2023-02-11 03:53:59,168 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-11 03:53:59,428 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0 2023-02-11 03:53:59,442 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 03:53:59,442 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544, deleteOnExit=true 2023-02-11 03:53:59,443 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 03:53:59,443 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/test.cache.data in system properties and HBase conf 2023-02-11 03:53:59,444 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 03:53:59,444 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/hadoop.log.dir in system properties and HBase conf 2023-02-11 03:53:59,445 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 03:53:59,445 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 03:53:59,446 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 03:53:59,552 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 03:53:59,916 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 03:53:59,920 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 03:53:59,920 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 03:53:59,921 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 03:53:59,921 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 03:53:59,921 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 03:53:59,921 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 03:53:59,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 03:53:59,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 03:53:59,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 03:53:59,922 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/nfs.dump.dir in system properties and HBase conf 2023-02-11 03:53:59,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/java.io.tmpdir in system properties and HBase conf 2023-02-11 03:53:59,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 03:53:59,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 03:53:59,923 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 03:54:00,469 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 03:54:00,473 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 03:54:00,757 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 03:54:00,911 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 03:54:00,925 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:54:00,962 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:54:00,993 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/java.io.tmpdir/Jetty_localhost_41387_hdfs____6ewr5r/webapp 2023-02-11 03:54:01,146 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41387 2023-02-11 03:54:01,158 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 03:54:01,158 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 03:54:01,560 WARN [Listener at localhost/39467] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:54:01,662 WARN [Listener at localhost/39467] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 03:54:01,679 WARN [Listener at localhost/39467] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:54:01,685 INFO [Listener at localhost/39467] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:54:01,690 INFO [Listener at localhost/39467] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/java.io.tmpdir/Jetty_localhost_46311_datanode____o29irc/webapp 2023-02-11 03:54:01,785 INFO [Listener at localhost/39467] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46311 2023-02-11 03:54:02,084 WARN [Listener at localhost/34157] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:54:02,097 WARN [Listener at localhost/34157] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 03:54:02,104 WARN [Listener at localhost/34157] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:54:02,106 INFO [Listener at localhost/34157] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:54:02,113 INFO [Listener at localhost/34157] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/java.io.tmpdir/Jetty_localhost_36329_datanode____pj6f32/webapp 2023-02-11 03:54:02,238 INFO [Listener at localhost/34157] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36329 2023-02-11 03:54:02,246 WARN [Listener at localhost/44011] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:54:02,267 WARN [Listener at localhost/44011] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 03:54:02,271 WARN [Listener at localhost/44011] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:54:02,273 INFO [Listener at localhost/44011] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:54:02,282 INFO [Listener at localhost/44011] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/java.io.tmpdir/Jetty_localhost_36913_datanode____.hplj5/webapp 2023-02-11 03:54:02,382 INFO [Listener at localhost/44011] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36913 2023-02-11 03:54:02,393 WARN [Listener at localhost/45015] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:54:02,583 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x514d751073094156: Processing first storage report for DS-ce427996-6602-401a-b1fe-31886c9dae10 from datanode 328ac628-71ce-4e94-9ad2-68da08fa646b 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x514d751073094156: from storage DS-ce427996-6602-401a-b1fe-31886c9dae10 node DatanodeRegistration(127.0.0.1:43309, datanodeUuid=328ac628-71ce-4e94-9ad2-68da08fa646b, infoPort=33175, infoSecurePort=0, ipcPort=34157, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x160700ba7a0c9f04: Processing first storage report for DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13 from datanode 5cdfedec-baf9-457a-bd3c-897c5ed5a081 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x160700ba7a0c9f04: from storage DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13 node DatanodeRegistration(127.0.0.1:45405, datanodeUuid=5cdfedec-baf9-457a-bd3c-897c5ed5a081, infoPort=38357, infoSecurePort=0, ipcPort=45015, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe546db06dbd7b711: Processing first storage report for DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3 from datanode e31d1c49-fd88-49e6-ab6b-686fae2c4c65 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe546db06dbd7b711: from storage DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3 node DatanodeRegistration(127.0.0.1:35177, datanodeUuid=e31d1c49-fd88-49e6-ab6b-686fae2c4c65, infoPort=37079, infoSecurePort=0, ipcPort=44011, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x514d751073094156: Processing first storage report for DS-b2d25edd-326d-4d4b-8ee7-72d3da522292 from datanode 328ac628-71ce-4e94-9ad2-68da08fa646b 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x514d751073094156: from storage DS-b2d25edd-326d-4d4b-8ee7-72d3da522292 node DatanodeRegistration(127.0.0.1:43309, datanodeUuid=328ac628-71ce-4e94-9ad2-68da08fa646b, infoPort=33175, infoSecurePort=0, ipcPort=34157, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,585 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x160700ba7a0c9f04: Processing first storage report for DS-28ccf047-7ee5-44f3-9aee-106409ffd6a8 from datanode 5cdfedec-baf9-457a-bd3c-897c5ed5a081 2023-02-11 03:54:02,586 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x160700ba7a0c9f04: from storage DS-28ccf047-7ee5-44f3-9aee-106409ffd6a8 node DatanodeRegistration(127.0.0.1:45405, datanodeUuid=5cdfedec-baf9-457a-bd3c-897c5ed5a081, infoPort=38357, infoSecurePort=0, ipcPort=45015, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,586 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe546db06dbd7b711: Processing first storage report for DS-e30c6607-e3bc-413d-8cb4-4d78b42a904b from datanode e31d1c49-fd88-49e6-ab6b-686fae2c4c65 2023-02-11 03:54:02,586 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe546db06dbd7b711: from storage DS-e30c6607-e3bc-413d-8cb4-4d78b42a904b node DatanodeRegistration(127.0.0.1:35177, datanodeUuid=e31d1c49-fd88-49e6-ab6b-686fae2c4c65, infoPort=37079, infoSecurePort=0, ipcPort=44011, storageInfo=lv=-57;cid=testClusterID;nsid=2000760390;c=1676087640548), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 03:54:02,785 DEBUG [Listener at localhost/45015] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0 2023-02-11 03:54:02,848 INFO [Listener at localhost/45015] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/zookeeper_0, clientPort=55824, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 03:54:02,865 INFO [Listener at localhost/45015] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55824 2023-02-11 03:54:02,873 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:02,875 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:03,545 INFO [Listener at localhost/45015] util.FSUtils(479): Created version file at hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b with version=8 2023-02-11 03:54:03,545 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/hbase-staging 2023-02-11 03:54:03,838 INFO [Listener at localhost/45015] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 03:54:04,308 INFO [Listener at localhost/45015] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 03:54:04,341 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,342 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,342 INFO [Listener at localhost/45015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:54:04,343 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,343 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:54:04,492 INFO [Listener at localhost/45015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:54:04,562 DEBUG [Listener at localhost/45015] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 03:54:04,656 INFO [Listener at localhost/45015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36397 2023-02-11 03:54:04,666 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,668 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,688 INFO [Listener at localhost/45015] zookeeper.RecoverableZooKeeper(93): Process identifier=master:36397 connecting to ZooKeeper ensemble=127.0.0.1:55824 2023-02-11 03:54:04,727 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:363970x0, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:54:04,730 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:36397-0x1004183b4960000 connected 2023-02-11 03:54:04,754 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:54:04,755 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:54:04,759 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:54:04,766 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36397 2023-02-11 03:54:04,766 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36397 2023-02-11 03:54:04,767 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36397 2023-02-11 03:54:04,767 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36397 2023-02-11 03:54:04,767 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36397 2023-02-11 03:54:04,773 INFO [Listener at localhost/45015] master.HMaster(439): hbase.rootdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b, hbase.cluster.distributed=false 2023-02-11 03:54:04,839 INFO [Listener at localhost/45015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 03:54:04,840 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,840 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,840 INFO [Listener at localhost/45015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:54:04,840 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,840 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:54:04,845 INFO [Listener at localhost/45015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:54:04,848 INFO [Listener at localhost/45015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36661 2023-02-11 03:54:04,851 INFO [Listener at localhost/45015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 03:54:04,858 DEBUG [Listener at localhost/45015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 03:54:04,859 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,861 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,863 INFO [Listener at localhost/45015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36661 connecting to ZooKeeper ensemble=127.0.0.1:55824 2023-02-11 03:54:04,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:366610x0, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:54:04,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36661-0x1004183b4960001 connected 2023-02-11 03:54:04,869 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:54:04,871 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:54:04,872 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:54:04,872 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36661 2023-02-11 03:54:04,873 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36661 2023-02-11 03:54:04,873 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36661 2023-02-11 03:54:04,874 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36661 2023-02-11 03:54:04,874 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36661 2023-02-11 03:54:04,888 INFO [Listener at localhost/45015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 03:54:04,888 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,889 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,889 INFO [Listener at localhost/45015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:54:04,890 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,890 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:54:04,890 INFO [Listener at localhost/45015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:54:04,892 INFO [Listener at localhost/45015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33925 2023-02-11 03:54:04,892 INFO [Listener at localhost/45015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 03:54:04,893 DEBUG [Listener at localhost/45015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 03:54:04,894 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,896 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,898 INFO [Listener at localhost/45015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33925 connecting to ZooKeeper ensemble=127.0.0.1:55824 2023-02-11 03:54:04,902 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:339250x0, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:54:04,903 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:339250x0, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:54:04,904 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:339250x0, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:54:04,904 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33925-0x1004183b4960002 connected 2023-02-11 03:54:04,905 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:54:04,905 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33925 2023-02-11 03:54:04,906 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33925 2023-02-11 03:54:04,906 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33925 2023-02-11 03:54:04,906 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33925 2023-02-11 03:54:04,907 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33925 2023-02-11 03:54:04,919 INFO [Listener at localhost/45015] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-11 03:54:04,920 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,920 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,920 INFO [Listener at localhost/45015] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:54:04,920 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:54:04,920 INFO [Listener at localhost/45015] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:54:04,921 INFO [Listener at localhost/45015] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:54:04,922 INFO [Listener at localhost/45015] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36767 2023-02-11 03:54:04,923 INFO [Listener at localhost/45015] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 03:54:04,924 DEBUG [Listener at localhost/45015] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 03:54:04,925 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,927 INFO [Listener at localhost/45015] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:04,929 INFO [Listener at localhost/45015] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36767 connecting to ZooKeeper ensemble=127.0.0.1:55824 2023-02-11 03:54:04,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:367670x0, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:54:04,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:36767-0x1004183b4960003 connected 2023-02-11 03:54:04,934 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:54:04,934 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:54:04,935 DEBUG [Listener at localhost/45015] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:54:04,936 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36767 2023-02-11 03:54:04,936 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36767 2023-02-11 03:54:04,936 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36767 2023-02-11 03:54:04,937 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36767 2023-02-11 03:54:04,937 DEBUG [Listener at localhost/45015] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36767 2023-02-11 03:54:04,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:04,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 03:54:04,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:04,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:54:04,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:54:04,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:54:04,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:54:04,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:04,969 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 03:54:04,971 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,36397,1676087643676 from backup master directory 2023-02-11 03:54:04,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 03:54:04,974 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:04,974 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 03:54:04,975 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:54:04,975 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:04,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 03:54:04,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 03:54:05,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/hbase.id with ID: 00923770-9827-42ed-9c22-5aea9c36eec9 2023-02-11 03:54:05,143 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:54:05,162 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:05,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x10c3821d to 127.0.0.1:55824 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:05,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6cece737, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:05,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 03:54:05,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 03:54:05,295 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 03:54:05,296 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 03:54:05,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:54:05,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:54:05,303 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:54:05,337 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store-tmp 2023-02-11 03:54:05,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:05,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 03:54:05,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:54:05,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:54:05,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 03:54:05,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:54:05,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:54:05,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676087645400Disabling compacts and flushes for region at 1676087645400Disabling writes for close at 1676087645401 (+1 ms)Writing region close event to WAL at 1676087645401Closed at 1676087645401 2023-02-11 03:54:05,402 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/WALs/jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:05,426 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36397%2C1676087643676, suffix=, logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/WALs/jenkins-hbase4.apache.org,36397,1676087643676, archiveDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/oldWALs, maxLogs=10 2023-02-11 03:54:05,485 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK] 2023-02-11 03:54:05,485 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK] 2023-02-11 03:54:05,485 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK] 2023-02-11 03:54:05,493 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:54:05,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/WALs/jenkins-hbase4.apache.org,36397,1676087643676/jenkins-hbase4.apache.org%2C36397%2C1676087643676.1676087645439 2023-02-11 03:54:05,576 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK], DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK], DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK]] 2023-02-11 03:54:05,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:05,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:05,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,657 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,665 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 03:54:05,690 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 03:54:05,703 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:05,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,727 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:54:05,731 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:05,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59350007, jitterRate=-0.11561597883701324}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:05,733 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676087645620Initializing all the Stores at 1676087645623 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087645624 (+1 ms)Cleaning up temporary data from old regions at 1676087645719 (+95 ms)Cleaning up detritus from prior splits at 1676087645720 (+1 ms)Region opened successfully at 1676087645732 (+12 ms) 2023-02-11 03:54:05,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 03:54:05,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:05,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:05,756 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:05,758 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-11 03:54:05,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-02-11 03:54:05,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 03:54:05,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 03:54:05,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 03:54:05,847 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 03:54:05,851 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 03:54:05,852 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 03:54:05,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 03:54:05,862 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 03:54:05,865 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:05,866 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 03:54:05,867 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 03:54:05,878 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 03:54:05,882 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:54:05,883 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:54:05,883 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:05,883 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:54:05,882 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:54:05,885 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,36397,1676087643676, sessionid=0x1004183b4960000, setting cluster-up flag (Was=false) 2023-02-11 03:54:05,904 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:05,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 03:54:05,911 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:05,918 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:05,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 03:54:05,926 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:05,929 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.hbase-snapshot/.tmp 2023-02-11 03:54:05,941 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(952): ClusterId : 00923770-9827-42ed-9c22-5aea9c36eec9 2023-02-11 03:54:05,941 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(952): ClusterId : 00923770-9827-42ed-9c22-5aea9c36eec9 2023-02-11 03:54:05,941 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(952): ClusterId : 00923770-9827-42ed-9c22-5aea9c36eec9 2023-02-11 03:54:05,947 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 03:54:05,947 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 03:54:05,947 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 03:54:05,953 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 03:54:05,953 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 03:54:05,953 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 03:54:05,954 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 03:54:05,954 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 03:54:05,954 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 03:54:05,959 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 03:54:05,959 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 03:54:05,959 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 03:54:05,961 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ReadOnlyZKClient(139): Connect 0x1f9b5be4 to 127.0.0.1:55824 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:05,961 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ReadOnlyZKClient(139): Connect 0x77c8c135 to 127.0.0.1:55824 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:05,961 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ReadOnlyZKClient(139): Connect 0x4d2905d0 to 127.0.0.1:55824 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:05,970 DEBUG [RS:1;jenkins-hbase4:33925] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@115ca667, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:05,971 DEBUG [RS:1;jenkins-hbase4:33925] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15633094, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:54:05,971 DEBUG [RS:0;jenkins-hbase4:36661] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43eee205, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:05,971 DEBUG [RS:0;jenkins-hbase4:36661] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47d16ea0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:54:05,972 DEBUG [RS:2;jenkins-hbase4:36767] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@790f21bc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:05,972 DEBUG [RS:2;jenkins-hbase4:36767] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d30aa45, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:54:05,996 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:33925 2023-02-11 03:54:05,998 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:36767 2023-02-11 03:54:05,998 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36661 2023-02-11 03:54:06,001 INFO [RS:0;jenkins-hbase4:36661] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 03:54:06,001 INFO [RS:2;jenkins-hbase4:36767] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 03:54:06,001 INFO [RS:2;jenkins-hbase4:36767] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 03:54:06,001 INFO [RS:1;jenkins-hbase4:33925] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 03:54:06,002 INFO [RS:1;jenkins-hbase4:33925] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 03:54:06,002 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 03:54:06,001 INFO [RS:0;jenkins-hbase4:36661] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 03:54:06,002 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 03:54:06,002 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 03:54:06,005 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:33925, startcode=1676087644887 2023-02-11 03:54:06,005 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:36661, startcode=1676087644839 2023-02-11 03:54:06,005 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:36767, startcode=1676087644919 2023-02-11 03:54:06,026 DEBUG [RS:2;jenkins-hbase4:36767] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 03:54:06,028 DEBUG [RS:0;jenkins-hbase4:36661] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 03:54:06,026 DEBUG [RS:1;jenkins-hbase4:33925] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 03:54:06,094 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:06,097 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57947, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 03:54:06,097 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53429, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 03:54:06,097 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51415, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 03:54:06,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:54:06,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:54:06,109 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:54:06,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:54:06,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 03:54:06,110 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 03:54:06,112 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:54:06,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36397] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 03:54:06,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 03:54:06,113 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087676116 2023-02-11 03:54:06,118 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 03:54:06,135 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:06,136 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 03:54:06,137 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 03:54:06,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 03:54:06,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 03:54:06,146 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 03:54:06,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 03:54:06,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 03:54:06,154 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 03:54:06,156 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,156 WARN [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 03:54:06,156 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 03:54:06,154 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 03:54:06,157 WARN [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 03:54:06,157 WARN [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 03:54:06,158 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 03:54:06,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 03:54:06,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 03:54:06,167 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 03:54:06,168 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 03:54:06,170 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087646170,5,FailOnTimeoutGroup] 2023-02-11 03:54:06,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087646170,5,FailOnTimeoutGroup] 2023-02-11 03:54:06,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 03:54:06,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,175 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,227 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:06,229 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:06,229 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b 2023-02-11 03:54:06,260 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:33925, startcode=1676087644887 2023-02-11 03:54:06,260 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:36767, startcode=1676087644919 2023-02-11 03:54:06,261 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,36397,1676087643676 with isa=jenkins-hbase4.apache.org/172.31.14.131:36661, startcode=1676087644839 2023-02-11 03:54:06,262 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:06,267 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 03:54:06,269 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,270 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=36397] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,271 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/info 2023-02-11 03:54:06,272 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 03:54:06,274 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,275 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 03:54:06,277 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b 2023-02-11 03:54:06,278 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b 2023-02-11 03:54:06,277 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b 2023-02-11 03:54:06,278 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:39467 2023-02-11 03:54:06,278 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:39467 2023-02-11 03:54:06,278 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 03:54:06,278 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:39467 2023-02-11 03:54:06,278 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 03:54:06,278 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 03:54:06,279 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:54:06,280 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 03:54:06,280 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,281 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 03:54:06,283 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/table 2023-02-11 03:54:06,284 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 03:54:06,285 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,287 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740 2023-02-11 03:54:06,288 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740 2023-02-11 03:54:06,288 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:54:06,289 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ZKUtil(162): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,289 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ZKUtil(162): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,290 WARN [RS:2;jenkins-hbase4:36767] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:54:06,290 WARN [RS:1;jenkins-hbase4:33925] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:54:06,291 INFO [RS:2;jenkins-hbase4:36767] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:54:06,291 INFO [RS:1;jenkins-hbase4:33925] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:54:06,291 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ZKUtil(162): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,291 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1947): logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,291 WARN [RS:0;jenkins-hbase4:36661] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:54:06,291 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1947): logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,292 INFO [RS:0;jenkins-hbase4:36661] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:54:06,292 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1947): logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,293 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33925,1676087644887] 2023-02-11 03:54:06,293 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36767,1676087644919] 2023-02-11 03:54:06,293 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36661,1676087644839] 2023-02-11 03:54:06,294 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 03:54:06,302 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 03:54:06,307 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:06,309 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70163576, jitterRate=0.04551875591278076}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 03:54:06,309 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ZKUtil(162): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,309 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676087646262Initializing all the Stores at 1676087646264 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087646265 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087646266 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087646266Cleaning up temporary data from old regions at 1676087646290 (+24 ms)Cleaning up detritus from prior splits at 1676087646292 (+2 ms)Region opened successfully at 1676087646309 (+17 ms) 2023-02-11 03:54:06,309 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ZKUtil(162): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,309 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ZKUtil(162): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,310 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 03:54:06,310 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ZKUtil(162): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,310 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 03:54:06,311 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 03:54:06,311 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 03:54:06,311 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ZKUtil(162): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,311 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 03:54:06,311 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ZKUtil(162): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,311 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ZKUtil(162): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,311 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ZKUtil(162): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,312 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ZKUtil(162): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,318 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 03:54:06,320 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676087646310Disabling compacts and flushes for region at 1676087646310Disabling writes for close at 1676087646311 (+1 ms)Writing region close event to WAL at 1676087646318 (+7 ms)Closed at 1676087646318 2023-02-11 03:54:06,324 DEBUG [RS:2;jenkins-hbase4:36767] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 03:54:06,324 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 03:54:06,324 DEBUG [RS:0;jenkins-hbase4:36661] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 03:54:06,327 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:06,327 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 03:54:06,336 INFO [RS:1;jenkins-hbase4:33925] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 03:54:06,336 INFO [RS:0;jenkins-hbase4:36661] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 03:54:06,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 03:54:06,336 INFO [RS:2;jenkins-hbase4:36767] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 03:54:06,353 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 03:54:06,356 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 03:54:06,361 INFO [RS:2;jenkins-hbase4:36767] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 03:54:06,362 INFO [RS:0;jenkins-hbase4:36661] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 03:54:06,362 INFO [RS:1;jenkins-hbase4:33925] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 03:54:06,366 INFO [RS:2;jenkins-hbase4:36767] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:54:06,366 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,368 INFO [RS:1;jenkins-hbase4:33925] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:54:06,368 INFO [RS:0;jenkins-hbase4:36661] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:54:06,369 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 03:54:06,369 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,369 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,369 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 03:54:06,376 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 03:54:06,391 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,391 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,391 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,392 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,391 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,392 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:54:06,393 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:54:06,393 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,393 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:54:06,394 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:1;jenkins-hbase4:33925] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:0;jenkins-hbase4:36661] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,394 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,395 DEBUG [RS:2;jenkins-hbase4:36767] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:54:06,400 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,401 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,401 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,405 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,405 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,405 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,406 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,406 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,406 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,425 INFO [RS:0;jenkins-hbase4:36661] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 03:54:06,428 INFO [RS:2;jenkins-hbase4:36767] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 03:54:06,425 INFO [RS:1;jenkins-hbase4:33925] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 03:54:06,432 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36661,1676087644839-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,432 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33925,1676087644887-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,432 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36767,1676087644919-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:06,469 INFO [RS:2;jenkins-hbase4:36767] regionserver.Replication(203): jenkins-hbase4.apache.org,36767,1676087644919 started 2023-02-11 03:54:06,469 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36767,1676087644919, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36767, sessionid=0x1004183b4960003 2023-02-11 03:54:06,469 INFO [RS:0;jenkins-hbase4:36661] regionserver.Replication(203): jenkins-hbase4.apache.org,36661,1676087644839 started 2023-02-11 03:54:06,469 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 03:54:06,469 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,36661,1676087644839, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36661, sessionid=0x1004183b4960001 2023-02-11 03:54:06,470 DEBUG [RS:2;jenkins-hbase4:36767] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,470 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 03:54:06,470 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36767,1676087644919' 2023-02-11 03:54:06,470 DEBUG [RS:0;jenkins-hbase4:36661] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,471 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:06,471 INFO [RS:1;jenkins-hbase4:33925] regionserver.Replication(203): jenkins-hbase4.apache.org,33925,1676087644887 started 2023-02-11 03:54:06,471 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33925,1676087644887, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33925, sessionid=0x1004183b4960002 2023-02-11 03:54:06,471 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 03:54:06,471 DEBUG [RS:1;jenkins-hbase4:33925] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,471 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:06,471 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36661,1676087644839' 2023-02-11 03:54:06,472 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:06,471 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33925,1676087644887' 2023-02-11 03:54:06,472 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:06,473 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:06,473 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 03:54:06,473 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 03:54:06,473 DEBUG [RS:2;jenkins-hbase4:36767] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:06,473 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36767,1676087644919' 2023-02-11 03:54:06,473 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:54:06,473 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:06,474 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 03:54:06,474 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 03:54:06,474 DEBUG [RS:0;jenkins-hbase4:36661] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:06,475 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36661,1676087644839' 2023-02-11 03:54:06,475 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:54:06,475 DEBUG [RS:0;jenkins-hbase4:36661] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:54:06,476 DEBUG [RS:2;jenkins-hbase4:36767] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:54:06,475 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 03:54:06,476 DEBUG [RS:0;jenkins-hbase4:36661] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 03:54:06,476 INFO [RS:0;jenkins-hbase4:36661] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 03:54:06,476 DEBUG [RS:2;jenkins-hbase4:36767] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 03:54:06,476 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 03:54:06,476 INFO [RS:2;jenkins-hbase4:36767] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 03:54:06,476 INFO [RS:0;jenkins-hbase4:36661] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 03:54:06,477 DEBUG [RS:1;jenkins-hbase4:33925] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,477 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33925,1676087644887' 2023-02-11 03:54:06,478 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:54:06,477 INFO [RS:2;jenkins-hbase4:36767] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 03:54:06,479 DEBUG [RS:1;jenkins-hbase4:33925] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:54:06,479 DEBUG [RS:1;jenkins-hbase4:33925] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 03:54:06,480 INFO [RS:1;jenkins-hbase4:33925] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 03:54:06,480 INFO [RS:1;jenkins-hbase4:33925] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 03:54:06,508 DEBUG [jenkins-hbase4:36397] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-11 03:54:06,513 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:06,520 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:06,520 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:06,520 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:06,520 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:06,523 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33925,1676087644887, state=OPENING 2023-02-11 03:54:06,531 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 03:54:06,534 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:06,535 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 03:54:06,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:06,590 INFO [RS:2;jenkins-hbase4:36767] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36767%2C1676087644919, suffix=, logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36767,1676087644919, archiveDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs, maxLogs=32 2023-02-11 03:54:06,590 INFO [RS:0;jenkins-hbase4:36661] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36661%2C1676087644839, suffix=, logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36661,1676087644839, archiveDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs, maxLogs=32 2023-02-11 03:54:06,590 INFO [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33925%2C1676087644887, suffix=, logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,33925,1676087644887, archiveDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs, maxLogs=32 2023-02-11 03:54:06,629 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK] 2023-02-11 03:54:06,630 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK] 2023-02-11 03:54:06,630 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK] 2023-02-11 03:54:06,632 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK] 2023-02-11 03:54:06,632 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK] 2023-02-11 03:54:06,633 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK] 2023-02-11 03:54:06,634 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK] 2023-02-11 03:54:06,635 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK] 2023-02-11 03:54:06,640 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK] 2023-02-11 03:54:06,658 INFO [RS:2;jenkins-hbase4:36767] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36767,1676087644919/jenkins-hbase4.apache.org%2C36767%2C1676087644919.1676087646599 2023-02-11 03:54:06,658 INFO [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,33925,1676087644887/jenkins-hbase4.apache.org%2C33925%2C1676087644887.1676087646599 2023-02-11 03:54:06,658 INFO [RS:0;jenkins-hbase4:36661] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,36661,1676087644839/jenkins-hbase4.apache.org%2C36661%2C1676087644839.1676087646599 2023-02-11 03:54:06,658 DEBUG [RS:2;jenkins-hbase4:36767] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK], DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK], DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK]] 2023-02-11 03:54:06,660 DEBUG [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK], DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK], DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK]] 2023-02-11 03:54:06,660 DEBUG [RS:0;jenkins-hbase4:36661] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK], DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK], DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK]] 2023-02-11 03:54:06,722 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:06,724 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:54:06,727 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55744, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:54:06,741 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 03:54:06,742 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:54:06,745 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33925%2C1676087644887.meta, suffix=.meta, logDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,33925,1676087644887, archiveDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs, maxLogs=32 2023-02-11 03:54:06,769 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK] 2023-02-11 03:54:06,770 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK] 2023-02-11 03:54:06,770 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK] 2023-02-11 03:54:06,777 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/WALs/jenkins-hbase4.apache.org,33925,1676087644887/jenkins-hbase4.apache.org%2C33925%2C1676087644887.meta.1676087646747.meta 2023-02-11 03:54:06,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45405,DS-a2bbc9ba-8b71-4d02-bfa5-4c1aa7253c13,DISK], DatanodeInfoWithStorage[127.0.0.1:35177,DS-18fee251-ae43-4ff2-9ea5-bb12e08330a3,DISK], DatanodeInfoWithStorage[127.0.0.1:43309,DS-ce427996-6602-401a-b1fe-31886c9dae10,DISK]] 2023-02-11 03:54:06,777 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:06,783 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:06,785 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 03:54:06,801 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 03:54:06,804 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 03:54:06,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 03:54:06,808 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:06,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 03:54:06,809 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 03:54:06,812 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 03:54:06,813 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/info 2023-02-11 03:54:06,813 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/info 2023-02-11 03:54:06,814 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 03:54:06,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,815 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 03:54:06,816 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:54:06,816 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:54:06,817 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 03:54:06,818 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,818 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 03:54:06,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/table 2023-02-11 03:54:06,819 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/table 2023-02-11 03:54:06,820 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 03:54:06,820 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:06,822 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740 2023-02-11 03:54:06,825 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740 2023-02-11 03:54:06,829 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 03:54:06,831 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 03:54:06,833 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70280955, jitterRate=0.047267839312553406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 03:54:06,833 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676087646809Writing region info on filesystem at 1676087646809Initializing all the Stores at 1676087646811 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087646811Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087646811Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087646811Cleaning up temporary data from old regions at 1676087646827 (+16 ms)Cleaning up detritus from prior splits at 1676087646827Running coprocessor post-open hooks at 1676087646833 (+6 ms)Region opened successfully at 1676087646833 2023-02-11 03:54:06,843 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676087646714 2023-02-11 03:54:06,860 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 03:54:06,861 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 03:54:06,862 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33925,1676087644887, state=OPEN 2023-02-11 03:54:06,864 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 03:54:06,864 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 03:54:06,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 03:54:06,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33925,1676087644887 in 326 msec 2023-02-11 03:54:06,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 03:54:06,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 535 msec 2023-02-11 03:54:06,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 884 msec 2023-02-11 03:54:06,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676087644963, completionTime=-1 2023-02-11 03:54:06,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-11 03:54:06,881 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:06,950 DEBUG [hconnection-0x440c1d01-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:06,954 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55752, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:06,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:06,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087706974 2023-02-11 03:54:06,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087766974 2023-02-11 03:54:06,974 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 93 msec 2023-02-11 03:54:07,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36397,1676087643676-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:07,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36397,1676087643676-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:07,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36397,1676087643676-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:07,004 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:36397, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:07,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 03:54:07,011 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 03:54:07,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 03:54:07,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 03:54:07,031 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:07,034 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:07,036 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:07,059 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/hbase/namespace/af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,061 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/hbase/namespace/af122cb86300c9588537dca293c1ed53 empty. 2023-02-11 03:54:07,062 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/hbase/namespace/af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,062 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 03:54:07,099 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:07,101 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => af122cb86300c9588537dca293c1ed53, NAME => 'hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:07,124 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:07,124 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing af122cb86300c9588537dca293c1ed53, disabling compactions & flushes 2023-02-11 03:54:07,124 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,124 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,124 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. after waiting 0 ms 2023-02-11 03:54:07,124 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,125 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,125 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for af122cb86300c9588537dca293c1ed53: Waiting for close lock at 1676087647124Disabling compacts and flushes for region at 1676087647124Disabling writes for close at 1676087647124Writing region close event to WAL at 1676087647125 (+1 ms)Closed at 1676087647125 2023-02-11 03:54:07,129 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:07,144 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087647132"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087647132"}]},"ts":"1676087647132"} 2023-02-11 03:54:07,173 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:07,174 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:07,179 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087647175"}]},"ts":"1676087647175"} 2023-02-11 03:54:07,184 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 03:54:07,189 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:07,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:07,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:07,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:07,190 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:07,193 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=af122cb86300c9588537dca293c1ed53, ASSIGN}] 2023-02-11 03:54:07,197 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=af122cb86300c9588537dca293c1ed53, ASSIGN 2023-02-11 03:54:07,199 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=af122cb86300c9588537dca293c1ed53, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:07,352 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:07,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=af122cb86300c9588537dca293c1ed53, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:07,353 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087647353"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087647353"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087647353"}]},"ts":"1676087647353"} 2023-02-11 03:54:07,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure af122cb86300c9588537dca293c1ed53, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:07,519 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,520 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => af122cb86300c9588537dca293c1ed53, NAME => 'hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:07,521 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:07,522 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,522 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:07,522 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,522 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,524 INFO [StoreOpener-af122cb86300c9588537dca293c1ed53-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,526 DEBUG [StoreOpener-af122cb86300c9588537dca293c1ed53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/info 2023-02-11 03:54:07,527 DEBUG [StoreOpener-af122cb86300c9588537dca293c1ed53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/info 2023-02-11 03:54:07,527 INFO [StoreOpener-af122cb86300c9588537dca293c1ed53-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region af122cb86300c9588537dca293c1ed53 columnFamilyName info 2023-02-11 03:54:07,528 INFO [StoreOpener-af122cb86300c9588537dca293c1ed53-1] regionserver.HStore(310): Store=af122cb86300c9588537dca293c1ed53/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:07,531 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,532 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,537 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for af122cb86300c9588537dca293c1ed53 2023-02-11 03:54:07,541 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:07,542 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened af122cb86300c9588537dca293c1ed53; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68295690, jitterRate=0.01768508553504944}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:07,542 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for af122cb86300c9588537dca293c1ed53: Running coprocessor pre-open hook at 1676087647522Writing region info on filesystem at 1676087647522Initializing all the Stores at 1676087647524 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087647524Cleaning up temporary data from old regions at 1676087647533 (+9 ms)Cleaning up detritus from prior splits at 1676087647533Running coprocessor post-open hooks at 1676087647542 (+9 ms)Region opened successfully at 1676087647542 2023-02-11 03:54:07,546 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53., pid=6, masterSystemTime=1676087647513 2023-02-11 03:54:07,549 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,549 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:54:07,552 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=af122cb86300c9588537dca293c1ed53, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:07,552 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087647551"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087647551"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087647551"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087647551"}]},"ts":"1676087647551"} 2023-02-11 03:54:07,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 03:54:07,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure af122cb86300c9588537dca293c1ed53, server=jenkins-hbase4.apache.org,33925,1676087644887 in 198 msec 2023-02-11 03:54:07,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 03:54:07,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=af122cb86300c9588537dca293c1ed53, ASSIGN in 370 msec 2023-02-11 03:54:07,568 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:07,568 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087647568"}]},"ts":"1676087647568"} 2023-02-11 03:54:07,571 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 03:54:07,574 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:07,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 552 msec 2023-02-11 03:54:07,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 03:54:07,640 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:54:07,640 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:07,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:07,732 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:54:07,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 69 msec 2023-02-11 03:54:07,746 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:07,758 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:54:07,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-11 03:54:07,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 03:54:07,776 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 03:54:07,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.801sec 2023-02-11 03:54:07,780 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 03:54:07,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 03:54:07,782 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 03:54:07,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36397,1676087643676-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 03:54:07,785 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36397,1676087643676-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 03:54:07,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 03:54:07,847 DEBUG [Listener at localhost/45015] zookeeper.ReadOnlyZKClient(139): Connect 0x328d140f to 127.0.0.1:55824 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:07,853 DEBUG [Listener at localhost/45015] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37e887de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:07,867 DEBUG [hconnection-0x5adffcfe-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:07,878 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55758, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:07,887 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:54:07,915 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=359, OpenFileDescriptor=696, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=169, AvailableMemoryMB=3463 2023-02-11 03:54:07,919 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:07,929 DEBUG [Listener at localhost/45015] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:07,932 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50682, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:07,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:07,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:54:07,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:07,953 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:07,955 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:07,957 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:07,960 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:07,961 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e empty. 2023-02-11 03:54:07,963 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:07,963 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 03:54:07,984 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:07,989 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => d8aa82cd827bdff360f5ba5a4ae4262e, NAME => 'testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing d8aa82cd827bdff360f5ba5a4ae4262e, disabling compactions & flushes 2023-02-11 03:54:08,011 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. after waiting 0 ms 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,011 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,011 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for d8aa82cd827bdff360f5ba5a4ae4262e: Waiting for close lock at 1676087648011Disabling compacts and flushes for region at 1676087648011Disabling writes for close at 1676087648011Writing region close event to WAL at 1676087648011Closed at 1676087648011 2023-02-11 03:54:08,016 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:08,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087648017"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087648017"}]},"ts":"1676087648017"} 2023-02-11 03:54:08,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:08,021 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:08,022 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087648022"}]},"ts":"1676087648022"} 2023-02-11 03:54:08,024 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-11 03:54:08,028 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:08,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:08,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:08,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:08,030 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:08,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, ASSIGN}] 2023-02-11 03:54:08,033 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, ASSIGN 2023-02-11 03:54:08,034 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:08,185 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:08,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d8aa82cd827bdff360f5ba5a4ae4262e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087648186"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087648186"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087648186"}]},"ts":"1676087648186"} 2023-02-11 03:54:08,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d8aa82cd827bdff360f5ba5a4ae4262e, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:08,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d8aa82cd827bdff360f5ba5a4ae4262e, NAME => 'testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:08,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:08,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:08,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,351 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,354 DEBUG [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1 2023-02-11 03:54:08,354 DEBUG [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1 2023-02-11 03:54:08,354 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8aa82cd827bdff360f5ba5a4ae4262e columnFamilyName cf1 2023-02-11 03:54:08,355 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] regionserver.HStore(310): Store=d8aa82cd827bdff360f5ba5a4ae4262e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:08,355 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,357 DEBUG [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2 2023-02-11 03:54:08,357 DEBUG [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2 2023-02-11 03:54:08,358 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8aa82cd827bdff360f5ba5a4ae4262e columnFamilyName cf2 2023-02-11 03:54:08,358 INFO [StoreOpener-d8aa82cd827bdff360f5ba5a4ae4262e-1] regionserver.HStore(310): Store=d8aa82cd827bdff360f5ba5a4ae4262e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:08,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:08,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:08,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:08,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d8aa82cd827bdff360f5ba5a4ae4262e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65532782, jitterRate=-0.023485451936721802}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:08,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d8aa82cd827bdff360f5ba5a4ae4262e: Running coprocessor pre-open hook at 1676087648350Writing region info on filesystem at 1676087648350Initializing all the Stores at 1676087648351 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087648351Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087648351Cleaning up temporary data from old regions at 1676087648362 (+11 ms)Cleaning up detritus from prior splits at 1676087648363 (+1 ms)Running coprocessor post-open hooks at 1676087648369 (+6 ms)Region opened successfully at 1676087648369 2023-02-11 03:54:08,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e., pid=11, masterSystemTime=1676087648343 2023-02-11 03:54:08,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d8aa82cd827bdff360f5ba5a4ae4262e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,375 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087648374"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087648374"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087648374"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087648374"}]},"ts":"1676087648374"} 2023-02-11 03:54:08,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-11 03:54:08,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d8aa82cd827bdff360f5ba5a4ae4262e, server=jenkins-hbase4.apache.org,33925,1676087644887 in 189 msec 2023-02-11 03:54:08,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-11 03:54:08,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, ASSIGN in 352 msec 2023-02-11 03:54:08,386 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:08,387 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087648386"}]},"ts":"1676087648386"} 2023-02-11 03:54:08,389 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-11 03:54:08,391 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:08,394 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 441 msec 2023-02-11 03:54:08,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:08,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,487 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:08,487 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,487 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:08,487 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:08,488 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,488 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,490 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,490 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,490 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,490 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,491 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,491 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,496 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,496 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,496 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,498 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,499 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:08,499 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,499 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:08,499 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:08,500 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:54:08,500 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,502 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,502 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,572 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/992de676a4a246fa9842792f58c4a9f5 2023-02-11 03:54:08,637 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/1f690259d6054dbdb5a5f81f8dc781a7 2023-02-11 03:54:08,647 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/992de676a4a246fa9842792f58c4a9f5 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5 2023-02-11 03:54:08,658 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:08,661 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/1f690259d6054dbdb5a5f81f8dc781a7 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7 2023-02-11 03:54:08,669 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:08,671 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 171ms, sequenceid=6, compaction requested=false 2023-02-11 03:54:08,672 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:08,675 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:08,675 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,675 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,677 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,677 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,678 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,678 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,679 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,679 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:08,682 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:08,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6685c95e[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:08,683 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,688 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:08,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,689 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:08,692 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,692 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,693 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,694 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:08,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:08,699 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:08,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:08,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:08,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,699 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:08,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,702 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,703 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,703 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,706 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,706 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,707 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,719 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,720 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,720 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,720 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,720 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,721 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,721 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,722 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:08,723 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:08,823 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:08,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:08,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:08,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:08,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,857 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:08,857 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,858 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:08,858 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:08,859 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,859 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,860 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,861 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,861 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,862 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:08,863 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:08,863 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,863 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,864 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:08,865 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:08,865 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,866 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:08,866 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:08,866 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,867 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,868 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,886 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/ae86c18faa3c4a18847accff224982b1 2023-02-11 03:54:08,911 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/fa9db0c1e40e4c55a0944646b81f4495 2023-02-11 03:54:08,920 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/ae86c18faa3c4a18847accff224982b1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1 2023-02-11 03:54:08,928 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:08,930 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/fa9db0c1e40e4c55a0944646b81f4495 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495 2023-02-11 03:54:08,938 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:08,939 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 73ms, sequenceid=19, compaction requested=false 2023-02-11 03:54:08,939 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:08,940 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:08,940 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,941 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:08,941 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:08,942 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,943 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,943 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:08,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14a94cb6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:08,945 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:08,945 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,948 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,948 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:08,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:08,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:08,948 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:08,949 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:08,951 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,951 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,951 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,952 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,952 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:08,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:08,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:08,958 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:08,958 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:08,958 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:08,958 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,958 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:08,959 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,959 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,960 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,960 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,960 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,961 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,962 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,964 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,965 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,965 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:08,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:08,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,976 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,977 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:08,977 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,976 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,977 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:08,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:08,978 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,078 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,102 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,105 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,105 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:09,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:09,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,108 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,109 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,109 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,111 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,111 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,113 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,113 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,112 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,113 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:09,113 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,113 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:09,115 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,116 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,133 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/d2ceac9169eb4f1e88b98db00e23ba7a 2023-02-11 03:54:09,161 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/93152a39526b49d4920aa27f6ff751dd 2023-02-11 03:54:09,170 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/d2ceac9169eb4f1e88b98db00e23ba7a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a 2023-02-11 03:54:09,179 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:09,180 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/93152a39526b49d4920aa27f6ff751dd as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd 2023-02-11 03:54:09,192 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:09,193 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 80ms, sequenceid=32, compaction requested=true 2023-02-11 03:54:09,194 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:09,194 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,194 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,195 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,197 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,198 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,198 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:09,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69a7abdc[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,201 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:09,201 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,202 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,202 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,202 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:09,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,203 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,203 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:09,207 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,207 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,207 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,207 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,208 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,208 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,208 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,208 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,208 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:09,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,213 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:09,213 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,213 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,213 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,213 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,216 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,218 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,220 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,220 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,221 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,224 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,225 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:09,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,225 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,326 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,351 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,354 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,354 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,354 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:09,355 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:09,355 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,355 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,357 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,357 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,358 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,358 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,359 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,359 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,359 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,360 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,361 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,362 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,362 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,362 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,362 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,362 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:09,363 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:09,364 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,385 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/7dfd7cf17cb349ee9092cd3789704aaa 2023-02-11 03:54:09,415 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/49fac48c7c0b4d4f84bd2ecf7377993d 2023-02-11 03:54:09,423 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/7dfd7cf17cb349ee9092cd3789704aaa as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa 2023-02-11 03:54:09,431 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:09,432 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/49fac48c7c0b4d4f84bd2ecf7377993d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d 2023-02-11 03:54:09,440 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:09,441 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 79ms, sequenceid=45, compaction requested=true 2023-02-11 03:54:09,441 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:09,441 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,441 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,442 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,442 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,442 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,442 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,447 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,447 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,447 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,447 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:09,450 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@506747c2[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,450 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:09,450 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,451 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:09,451 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,452 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,452 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,452 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:09,454 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,454 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,454 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,455 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,455 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,456 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:09,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,460 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:09,460 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,460 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,460 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,460 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,462 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,463 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,464 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,467 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,467 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,467 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,471 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,472 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,471 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,472 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,472 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,472 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,573 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,598 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:09,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:09,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,604 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,604 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,611 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,611 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,611 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,612 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,612 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,613 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,613 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,613 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,615 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,615 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,615 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,615 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:09,615 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:09,617 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,617 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,617 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,618 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,618 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,618 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,635 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/20c0f6e5e896494f820a5629468777ec 2023-02-11 03:54:09,663 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/a8a1e4d251bb42a0af9b3e0262266395 2023-02-11 03:54:09,671 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/20c0f6e5e896494f820a5629468777ec as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec 2023-02-11 03:54:09,678 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:09,679 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/a8a1e4d251bb42a0af9b3e0262266395 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395 2023-02-11 03:54:09,685 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:09,687 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 72ms, sequenceid=58, compaction requested=true 2023-02-11 03:54:09,687 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:09,687 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,687 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,690 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18e0bba9[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,692 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:09,692 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,695 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,695 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,696 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:09,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,698 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,698 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,698 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,698 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,699 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:09,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,704 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:09,704 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,704 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,704 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,704 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,707 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,707 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,708 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,709 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,709 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,711 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,711 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,711 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,718 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,718 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:09,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,718 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,719 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,819 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,844 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,844 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,844 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-11 03:54:09,844 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-11 03:54:09,845 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,845 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,847 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,847 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,847 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,849 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-11 03:54:09,849 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,849 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,849 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-11 03:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,849 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,850 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,851 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-11 03:54:09,851 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,851 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,851 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,851 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,851 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. started... 2023-02-11 03:54:09,851 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,852 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:09,853 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,854 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,869 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/afa7799ae0b04462964f3b33401bd902 2023-02-11 03:54:09,889 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/9b4481fe246846289bfc30c2ce70c65e 2023-02-11 03:54:09,899 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/afa7799ae0b04462964f3b33401bd902 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902 2023-02-11 03:54:09,906 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:09,907 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/9b4481fe246846289bfc30c2ce70c65e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e 2023-02-11 03:54:09,914 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:09,915 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 63ms, sequenceid=71, compaction requested=true 2023-02-11 03:54:09,915 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d8aa82cd827bdff360f5ba5a4ae4262e: 2023-02-11 03:54:09,915 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:09,915 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-11 03:54:09,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-11 03:54:09,918 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,918 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,919 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-11 03:54:09,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46f288ff[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,921 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:09,921 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-11 03:54:09,922 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:09,923 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:09,925 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,925 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,925 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-11 03:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,926 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,927 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-11 03:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:09,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,931 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-11 03:54:09,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,931 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,933 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,940 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-11 03:54:09,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,941 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,941 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:09,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:09,942 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:09,943 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:09,943 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-11 03:54:10,042 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:10,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:10,065 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:10,071 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:10,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:10,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:10,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:10,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, UNASSIGN}] 2023-02-11 03:54:10,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, UNASSIGN 2023-02-11 03:54:10,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d8aa82cd827bdff360f5ba5a4ae4262e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:10,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087650082"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087650082"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087650082"}]},"ts":"1676087650082"} 2023-02-11 03:54:10,084 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure d8aa82cd827bdff360f5ba5a4ae4262e, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:10,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d8aa82cd827bdff360f5ba5a4ae4262e, disabling compactions & flushes 2023-02-11 03:54:10,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:10,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:10,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. after waiting 0 ms 2023-02-11 03:54:10,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:10,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d8aa82cd827bdff360f5ba5a4ae4262e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:54:10,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/8769ac44db9f45aa97a9e53e23059994 2023-02-11 03:54:10,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/a8fc2aca433b422694754c7a0c35d3ab 2023-02-11 03:54:10,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf1/8769ac44db9f45aa97a9e53e23059994 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994 2023-02-11 03:54:10,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:10,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.tmp/cf2/a8fc2aca433b422694754c7a0c35d3ab as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab 2023-02-11 03:54:10,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:10,310 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 65ms, sequenceid=82, compaction requested=true 2023-02-11 03:54:10,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:54:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:10,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. 2023-02-11 03:54:10,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d8aa82cd827bdff360f5ba5a4ae4262e: Waiting for close lock at 1676087650244Running coprocessor pre-close hooks at 1676087650244Disabling compacts and flushes for region at 1676087650244Disabling writes for close at 1676087650244Obtaining lock to block concurrent updates at 1676087650244Preparing flush snapshotting stores in d8aa82cd827bdff360f5ba5a4ae4262e at 1676087650244Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087650245 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. at 1676087650245Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf1: creating writer at 1676087650246 (+1 ms)Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf1: appending metadata at 1676087650249 (+3 ms)Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf1: closing flushed file at 1676087650249Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf2: creating writer at 1676087650268 (+19 ms)Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf2: appending metadata at 1676087650271 (+3 ms)Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf2: closing flushed file at 1676087650271Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf1: reopening flushed file at 1676087650293 (+22 ms)Flushing d8aa82cd827bdff360f5ba5a4ae4262e/cf2: reopening flushed file at 1676087650302 (+9 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d8aa82cd827bdff360f5ba5a4ae4262e in 65ms, sequenceid=82, compaction requested=true at 1676087650310 (+8 ms)Writing region close event to WAL at 1676087650325 (+15 ms)Running coprocessor post-close hooks at 1676087650331 (+6 ms)Closed at 1676087650331 2023-02-11 03:54:10,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,335 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d8aa82cd827bdff360f5ba5a4ae4262e, regionState=CLOSED 2023-02-11 03:54:10,336 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087650335"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087650335"}]},"ts":"1676087650335"} 2023-02-11 03:54:10,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-11 03:54:10,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure d8aa82cd827bdff360f5ba5a4ae4262e, server=jenkins-hbase4.apache.org,33925,1676087644887 in 254 msec 2023-02-11 03:54:10,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-11 03:54:10,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d8aa82cd827bdff360f5ba5a4ae4262e, UNASSIGN in 260 msec 2023-02-11 03:54:10,360 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=d8aa82cd827bdff360f5ba5a4ae4262e, threads=14 2023-02-11 03:54:10,361 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,361 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,362 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,363 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,363 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,365 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,365 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,365 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,365 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,391 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395, top=true 2023-02-11 03:54:10,395 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec, top=true 2023-02-11 03:54:10,396 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e, top=true 2023-02-11 03:54:10,397 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994, top=true 2023-02-11 03:54:10,400 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902, top=true 2023-02-11 03:54:10,401 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab, top=true 2023-02-11 03:54:10,446 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,446 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,447 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,447 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395 for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902 for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994 for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,452 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,464 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/.splits/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e for child: 6ecb2d71d524c8ebc2c51ae55e3c1ca2, parent: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,464 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,544 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,557 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,560 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,560 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,563 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,564 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1 for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,565 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,966 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d for region: d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:10,966 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region d8aa82cd827bdff360f5ba5a4ae4262e Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:11,019 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:11,022 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:11,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676087651024"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676087651024"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676087651024"}]},"ts":"1676087651024"} 2023-02-11 03:54:11,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087651024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087651024"}]},"ts":"1676087651024"} 2023-02-11 03:54:11,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087651024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087651024"}]},"ts":"1676087651024"} 2023-02-11 03:54:11,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, ASSIGN}] 2023-02-11 03:54:11,076 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, ASSIGN 2023-02-11 03:54:11,076 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:11,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, ASSIGN 2023-02-11 03:54:11,078 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:11,227 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:54:11,228 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=6ecb2d71d524c8ebc2c51ae55e3c1ca2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:11,229 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651228"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087651228"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087651228"}]},"ts":"1676087651228"} 2023-02-11 03:54:11,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 6ecb2d71d524c8ebc2c51ae55e3c1ca2, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:11,235 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0f490700d6877b22c5abe6f9e427a663, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:11,235 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651235"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087651235"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087651235"}]},"ts":"1676087651235"} 2023-02-11 03:54:11,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 0f490700d6877b22c5abe6f9e427a663, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:11,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0f490700d6877b22c5abe6f9e427a663, NAME => 'testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.', STARTKEY => '', ENDKEY => '26'} 2023-02-11 03:54:11,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:11,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:11,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,391 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,392 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1 2023-02-11 03:54:11,392 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1 2023-02-11 03:54:11,393 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f490700d6877b22c5abe6f9e427a663 columnFamilyName cf1 2023-02-11 03:54:11,426 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-bottom 2023-02-11 03:54:11,433 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-bottom 2023-02-11 03:54:11,439 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-bottom 2023-02-11 03:54:11,447 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-bottom 2023-02-11 03:54:11,447 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(310): Store=0f490700d6877b22c5abe6f9e427a663/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:11,447 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,448 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2 2023-02-11 03:54:11,448 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2 2023-02-11 03:54:11,449 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0f490700d6877b22c5abe6f9e427a663 columnFamilyName cf2 2023-02-11 03:54:11,463 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-bottom 2023-02-11 03:54:11,469 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-bottom 2023-02-11 03:54:11,476 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-bottom 2023-02-11 03:54:11,483 DEBUG [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-bottom 2023-02-11 03:54:11,483 INFO [StoreOpener-0f490700d6877b22c5abe6f9e427a663-1] regionserver.HStore(310): Store=0f490700d6877b22c5abe6f9e427a663/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:11,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:11,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:11,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0f490700d6877b22c5abe6f9e427a663; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60291658, jitterRate=-0.1015842854976654}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:11,490 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0f490700d6877b22c5abe6f9e427a663: Running coprocessor pre-open hook at 1676087651389Writing region info on filesystem at 1676087651389Initializing all the Stores at 1676087651390 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087651390Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087651390Cleaning up temporary data from old regions at 1676087651487 (+97 ms)Cleaning up detritus from prior splits at 1676087651487Running coprocessor post-open hooks at 1676087651490 (+3 ms)Region opened successfully at 1676087651490 2023-02-11 03:54:11,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663., pid=18, masterSystemTime=1676087651384 2023-02-11 03:54:11,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:11,494 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:54:11,494 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:54:11,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,497 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6ecb2d71d524c8ebc2c51ae55e3c1ca2, NAME => 'testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.', STARTKEY => '26', ENDKEY => ''} 2023-02-11 03:54:11,497 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,497 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): 0f490700d6877b22c5abe6f9e427a663/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:11,497 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): 0f490700d6877b22c5abe6f9e427a663/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:11,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:11,497 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f490700d6877b22c5abe6f9e427a663/cf1 in testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0f490700d6877b22c5abe6f9e427a663, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:11,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,497 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0f490700d6877b22c5abe6f9e427a663/cf2 in testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:11,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:11,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651497"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087651497"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087651497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087651497"}]},"ts":"1676087651497"} 2023-02-11 03:54:11,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,498 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/.tmp, totalSize=19.9 K 2023-02-11 03:54:11,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,499 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/.tmp, totalSize=19.9 K 2023-02-11 03:54:11,500 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087648459 2023-02-11 03:54:11,501 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087648459 2023-02-11 03:54:11,502 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087648831 2023-02-11 03:54:11,502 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087648831 2023-02-11 03:54:11,502 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,502 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087649081 2023-02-11 03:54:11,503 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087649081 2023-02-11 03:54:11,503 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087649329 2023-02-11 03:54:11,504 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1 2023-02-11 03:54:11,504 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087649329 2023-02-11 03:54:11,504 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1 2023-02-11 03:54:11,505 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ecb2d71d524c8ebc2c51ae55e3c1ca2 columnFamilyName cf1 2023-02-11 03:54:11,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-11 03:54:11,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 0f490700d6877b22c5abe6f9e427a663, server=jenkins-hbase4.apache.org,33925,1676087644887 in 265 msec 2023-02-11 03:54:11,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, ASSIGN in 433 msec 2023-02-11 03:54:11,526 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0f490700d6877b22c5abe6f9e427a663#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:11,527 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-top 2023-02-11 03:54:11,527 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): 0f490700d6877b22c5abe6f9e427a663#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:11,536 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-top 2023-02-11 03:54:11,545 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-top 2023-02-11 03:54:11,558 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-top 2023-02-11 03:54:11,567 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/.tmp/cf1/6ae6ff276df949168bccfc903131253c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/6ae6ff276df949168bccfc903131253c 2023-02-11 03:54:11,569 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec 2023-02-11 03:54:11,575 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/.tmp/cf2/25a7cdf6d55c4a0aaa625425630cbecd as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/25a7cdf6d55c4a0aaa625425630cbecd 2023-02-11 03:54:11,576 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994 2023-02-11 03:54:11,581 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902 2023-02-11 03:54:11,581 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(310): Store=6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:11,581 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,583 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2 2023-02-11 03:54:11,583 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2 2023-02-11 03:54:11,583 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ecb2d71d524c8ebc2c51ae55e3c1ca2 columnFamilyName cf2 2023-02-11 03:54:11,586 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 0f490700d6877b22c5abe6f9e427a663/cf2 of 0f490700d6877b22c5abe6f9e427a663 into 25a7cdf6d55c4a0aaa625425630cbecd(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:11,586 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 0f490700d6877b22c5abe6f9e427a663/cf1 of 0f490700d6877b22c5abe6f9e427a663 into 6ae6ff276df949168bccfc903131253c(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:11,588 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f490700d6877b22c5abe6f9e427a663: 2023-02-11 03:54:11,588 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0f490700d6877b22c5abe6f9e427a663: 2023-02-11 03:54:11,588 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663., storeName=0f490700d6877b22c5abe6f9e427a663/cf1, priority=12, startTime=1676087651493; duration=0sec 2023-02-11 03:54:11,588 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663., storeName=0f490700d6877b22c5abe6f9e427a663/cf2, priority=12, startTime=1676087651493; duration=0sec 2023-02-11 03:54:11,589 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,589 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,600 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-top 2023-02-11 03:54:11,605 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-top 2023-02-11 03:54:11,610 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-top 2023-02-11 03:54:11,615 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-top 2023-02-11 03:54:11,620 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e 2023-02-11 03:54:11,625 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395 2023-02-11 03:54:11,630 DEBUG [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab 2023-02-11 03:54:11,630 INFO [StoreOpener-6ecb2d71d524c8ebc2c51ae55e3c1ca2-1] regionserver.HStore(310): Store=6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:11,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:11,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:11,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6ecb2d71d524c8ebc2c51ae55e3c1ca2; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66614741, jitterRate=-0.0073630064725875854}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:11,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6ecb2d71d524c8ebc2c51ae55e3c1ca2: Running coprocessor pre-open hook at 1676087651499Writing region info on filesystem at 1676087651499Initializing all the Stores at 1676087651501 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087651502 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087651502Cleaning up temporary data from old regions at 1676087651635 (+133 ms)Cleaning up detritus from prior splits at 1676087651635Running coprocessor post-open hooks at 1676087651638 (+3 ms)Region opened successfully at 1676087651638 2023-02-11 03:54:11,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2., pid=17, masterSystemTime=1676087651384 2023-02-11 03:54:11,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,641 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:11,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:11,641 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:11,641 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,641 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:11,641 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,641 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1 in testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,641 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:11,641 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2 in testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,642 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/.tmp, totalSize=34.9 K 2023-02-11 03:54:11,642 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/.tmp, totalSize=34.9 K 2023-02-11 03:54:11,642 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087648459 2023-02-11 03:54:11,643 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087648459 2023-02-11 03:54:11,643 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087648831 2023-02-11 03:54:11,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:11,644 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087648831 2023-02-11 03:54:11,644 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087649081 2023-02-11 03:54:11,644 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087649081 2023-02-11 03:54:11,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=6ecb2d71d524c8ebc2c51ae55e3c1ca2, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:11,645 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087651644"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087651644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087651644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087651644"}]},"ts":"1676087651644"} 2023-02-11 03:54:11,645 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087649329 2023-02-11 03:54:11,645 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087649329 2023-02-11 03:54:11,646 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087649576 2023-02-11 03:54:11,646 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087649576 2023-02-11 03:54:11,646 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087649822 2023-02-11 03:54:11,646 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087649822 2023-02-11 03:54:11,647 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087650045 2023-02-11 03:54:11,647 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087650045 2023-02-11 03:54:11,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-11 03:54:11,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 6ecb2d71d524c8ebc2c51ae55e3c1ca2, server=jenkins-hbase4.apache.org,33925,1676087644887 in 417 msec 2023-02-11 03:54:11,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-11 03:54:11,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, ASSIGN in 577 msec 2023-02-11 03:54:11,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 in 1.5890 sec 2023-02-11 03:54:11,674 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): 6ecb2d71d524c8ebc2c51ae55e3c1ca2#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:11,681 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6ecb2d71d524c8ebc2c51ae55e3c1ca2#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:11,698 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/.tmp/cf1/b9366f7d420748bd968eaf2f53c7cd94 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/b9366f7d420748bd968eaf2f53c7cd94 2023-02-11 03:54:11,698 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/.tmp/cf2/60014900e3214132a0d5a615783306be as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/60014900e3214132a0d5a615783306be 2023-02-11 03:54:11,706 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1 of 6ecb2d71d524c8ebc2c51ae55e3c1ca2 into b9366f7d420748bd968eaf2f53c7cd94(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:11,707 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6ecb2d71d524c8ebc2c51ae55e3c1ca2: 2023-02-11 03:54:11,707 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2., storeName=6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1, priority=9, startTime=1676087651640; duration=0sec 2023-02-11 03:54:11,707 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,710 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2 of 6ecb2d71d524c8ebc2c51ae55e3c1ca2 into 60014900e3214132a0d5a615783306be(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:11,710 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6ecb2d71d524c8ebc2c51ae55e3c1ca2: 2023-02-11 03:54:11,710 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2., storeName=6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2, priority=9, startTime=1676087651641; duration=0sec 2023-02-11 03:54:11,710 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:11,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33925] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:55758 deadline: 1676087711824, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e. is not online on jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:11,948 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 03:54:11,959 INFO [Listener at localhost/45015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 03:54:11,962 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-11 03:54:12,003 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:12,014 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,030 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087652030"}]},"ts":"1676087652030"} 2023-02-11 03:54:12,032 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-11 03:54:12,035 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-11 03:54:12,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:54:12,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, UNASSIGN}] 2023-02-11 03:54:12,038 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, UNASSIGN 2023-02-11 03:54:12,039 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6ecb2d71d524c8ebc2c51ae55e3c1ca2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:12,040 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087652039"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087652039"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087652039"}]},"ts":"1676087652039"} 2023-02-11 03:54:12,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 6ecb2d71d524c8ebc2c51ae55e3c1ca2, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:12,045 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, UNASSIGN 2023-02-11 03:54:12,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=0f490700d6877b22c5abe6f9e427a663, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:12,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087652046"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087652046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087652046"}]},"ts":"1676087652046"} 2023-02-11 03:54:12,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 0f490700d6877b22c5abe6f9e427a663, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:12,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:54:12,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:12,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6ecb2d71d524c8ebc2c51ae55e3c1ca2, disabling compactions & flushes 2023-02-11 03:54:12,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:12,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:12,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. after waiting 0 ms 2023-02-11 03:54:12,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:12,205 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994] to archive 2023-02-11 03:54:12,207 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:12,213 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,216 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,219 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,221 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,222 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-20c0f6e5e896494f820a5629468777ec 2023-02-11 03:54:12,224 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-afa7799ae0b04462964f3b33401bd902 2023-02-11 03:54:12,226 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-8769ac44db9f45aa97a9e53e23059994 2023-02-11 03:54:12,247 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab] to archive 2023-02-11 03:54:12,249 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:12,252 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,254 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,261 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,264 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,266 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8a1e4d251bb42a0af9b3e0262266395 2023-02-11 03:54:12,267 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-9b4481fe246846289bfc30c2ce70c65e 2023-02-11 03:54:12,270 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/testSplitTableRegionUnevenDaughter=d8aa82cd827bdff360f5ba5a4ae4262e-a8fc2aca433b422694754c7a0c35d3ab 2023-02-11 03:54:12,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:54:12,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:12,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2. 2023-02-11 03:54:12,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6ecb2d71d524c8ebc2c51ae55e3c1ca2: Waiting for close lock at 1676087652194Running coprocessor pre-close hooks at 1676087652194Disabling compacts and flushes for region at 1676087652194Disabling writes for close at 1676087652194Writing region close event to WAL at 1676087652274 (+80 ms)Running coprocessor post-close hooks at 1676087652287 (+13 ms)Closed at 1676087652288 (+1 ms) 2023-02-11 03:54:12,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:12,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:12,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0f490700d6877b22c5abe6f9e427a663, disabling compactions & flushes 2023-02-11 03:54:12,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:12,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:12,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. after waiting 0 ms 2023-02-11 03:54:12,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:12,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=6ecb2d71d524c8ebc2c51ae55e3c1ca2, regionState=CLOSED 2023-02-11 03:54:12,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087652297"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087652297"}]},"ts":"1676087652297"} 2023-02-11 03:54:12,311 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa-bottom] to archive 2023-02-11 03:54:12,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-11 03:54:12,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 6ecb2d71d524c8ebc2c51ae55e3c1ca2, server=jenkins-hbase4.apache.org,33925,1676087644887 in 259 msec 2023-02-11 03:54:12,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6ecb2d71d524c8ebc2c51ae55e3c1ca2, UNASSIGN in 277 msec 2023-02-11 03:54:12,317 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:12,324 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/992de676a4a246fa9842792f58c4a9f5.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,326 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/ae86c18faa3c4a18847accff224982b1.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,328 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/d2ceac9169eb4f1e88b98db00e23ba7a.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,331 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/7dfd7cf17cb349ee9092cd3789704aaa.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,334 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d-bottom] to archive 2023-02-11 03:54:12,336 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:12,337 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:12,341 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/1f690259d6054dbdb5a5f81f8dc781a7.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,342 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/fa9db0c1e40e4c55a0944646b81f4495.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:54:12,344 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/93152a39526b49d4920aa27f6ff751dd.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,346 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/49fac48c7c0b4d4f84bd2ecf7377993d.d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:54:12,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:12,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663. 2023-02-11 03:54:12,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0f490700d6877b22c5abe6f9e427a663: Waiting for close lock at 1676087652296Running coprocessor pre-close hooks at 1676087652296Disabling compacts and flushes for region at 1676087652296Disabling writes for close at 1676087652296Writing region close event to WAL at 1676087652353 (+57 ms)Running coprocessor post-close hooks at 1676087652366 (+13 ms)Closed at 1676087652366 2023-02-11 03:54:12,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:12,370 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=0f490700d6877b22c5abe6f9e427a663, regionState=CLOSED 2023-02-11 03:54:12,370 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087652370"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087652370"}]},"ts":"1676087652370"} 2023-02-11 03:54:12,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-11 03:54:12,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 0f490700d6877b22c5abe6f9e427a663, server=jenkins-hbase4.apache.org,33925,1676087644887 in 325 msec 2023-02-11 03:54:12,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-11 03:54:12,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0f490700d6877b22c5abe6f9e427a663, UNASSIGN in 340 msec 2023-02-11 03:54:12,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087652382"}]},"ts":"1676087652382"} 2023-02-11 03:54:12,385 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-11 03:54:12,388 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-11 03:54:12,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 368 msec 2023-02-11 03:54:12,415 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 03:54:12,416 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 03:54:12,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:54:12,645 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-11 03:54:12,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,658 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,660 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 03:54:12,667 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,667 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:12,667 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:12,671 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/recovered.edits] 2023-02-11 03:54:12,671 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/recovered.edits] 2023-02-11 03:54:12,671 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/recovered.edits] 2023-02-11 03:54:12,676 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/b9366f7d420748bd968eaf2f53c7cd94 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf1/b9366f7d420748bd968eaf2f53c7cd94 2023-02-11 03:54:12,676 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/20c0f6e5e896494f820a5629468777ec 2023-02-11 03:54:12,677 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/6ae6ff276df949168bccfc903131253c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf1/6ae6ff276df949168bccfc903131253c 2023-02-11 03:54:12,678 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/7dfd7cf17cb349ee9092cd3789704aaa 2023-02-11 03:54:12,679 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/60014900e3214132a0d5a615783306be to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/cf2/60014900e3214132a0d5a615783306be 2023-02-11 03:54:12,680 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/8769ac44db9f45aa97a9e53e23059994 2023-02-11 03:54:12,681 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/25a7cdf6d55c4a0aaa625425630cbecd to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/cf2/25a7cdf6d55c4a0aaa625425630cbecd 2023-02-11 03:54:12,682 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/992de676a4a246fa9842792f58c4a9f5 2023-02-11 03:54:12,684 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2/recovered.edits/90.seqid 2023-02-11 03:54:12,684 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/ae86c18faa3c4a18847accff224982b1 2023-02-11 03:54:12,685 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:12,685 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663/recovered.edits/90.seqid 2023-02-11 03:54:12,686 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/0f490700d6877b22c5abe6f9e427a663 2023-02-11 03:54:12,687 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/afa7799ae0b04462964f3b33401bd902 2023-02-11 03:54:12,688 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf1/d2ceac9169eb4f1e88b98db00e23ba7a 2023-02-11 03:54:12,691 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/1f690259d6054dbdb5a5f81f8dc781a7 2023-02-11 03:54:12,693 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/49fac48c7c0b4d4f84bd2ecf7377993d 2023-02-11 03:54:12,694 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/93152a39526b49d4920aa27f6ff751dd 2023-02-11 03:54:12,696 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/9b4481fe246846289bfc30c2ce70c65e 2023-02-11 03:54:12,698 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8a1e4d251bb42a0af9b3e0262266395 2023-02-11 03:54:12,699 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/a8fc2aca433b422694754c7a0c35d3ab 2023-02-11 03:54:12,701 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/cf2/fa9db0c1e40e4c55a0944646b81f4495 2023-02-11 03:54:12,704 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e/recovered.edits/85.seqid 2023-02-11 03:54:12,704 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionUnevenDaughter/d8aa82cd827bdff360f5ba5a4ae4262e 2023-02-11 03:54:12,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-11 03:54:12,707 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,729 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-11 03:54:12,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 03:54:12,772 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-11 03:54:12,773 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,773 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-11 03:54:12,774 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087652773"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:12,774 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087652773"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:12,774 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087652773"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:12,777 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:54:12,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d8aa82cd827bdff360f5ba5a4ae4262e, NAME => 'testSplitTableRegionUnevenDaughter,,1676087647951.d8aa82cd827bdff360f5ba5a4ae4262e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 0f490700d6877b22c5abe6f9e427a663, NAME => 'testSplitTableRegionUnevenDaughter,,1676087650065.0f490700d6877b22c5abe6f9e427a663.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 6ecb2d71d524c8ebc2c51ae55e3c1ca2, NAME => 'testSplitTableRegionUnevenDaughter,26,1676087650065.6ecb2d71d524c8ebc2c51ae55e3c1ca2.', STARTKEY => '26', ENDKEY => ''}] 2023-02-11 03:54:12,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-11 03:54:12,777 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087652777"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:12,779 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-11 03:54:12,781 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:12,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 130 msec 2023-02-11 03:54:12,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-11 03:54:12,964 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-11 03:54:12,981 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=389 (was 359) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33564 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43240 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33572 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33338 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:33925-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33586 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:58812 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43750 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:58918 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33268 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43442 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33294 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:58652 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43784 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43532 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33560 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33230 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=847 (was 696) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=243 (was 264), ProcessCount=168 (was 169), AvailableMemoryMB=3424 (was 3463) 2023-02-11 03:54:12,995 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=389, OpenFileDescriptor=847, MaxFileDescriptor=60000, SystemLoadAverage=243, ProcessCount=168, AvailableMemoryMB=3424 2023-02-11 03:54:12,995 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:13,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:13,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:13,005 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:13,007 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:13,008 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:13,010 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,010 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b empty. 2023-02-11 03:54:13,011 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,011 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 03:54:13,023 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:13,024 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 09e96c0d0642358ef90bd77ad1a2435b, NAME => 'testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 09e96c0d0642358ef90bd77ad1a2435b, disabling compactions & flushes 2023-02-11 03:54:13,034 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. after waiting 0 ms 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,034 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,034 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 09e96c0d0642358ef90bd77ad1a2435b: Waiting for close lock at 1676087653034Disabling compacts and flushes for region at 1676087653034Disabling writes for close at 1676087653034Writing region close event to WAL at 1676087653034Closed at 1676087653034 2023-02-11 03:54:13,037 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:13,038 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087653038"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087653038"}]},"ts":"1676087653038"} 2023-02-11 03:54:13,040 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:13,041 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:13,041 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087653041"}]},"ts":"1676087653041"} 2023-02-11 03:54:13,043 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-11 03:54:13,045 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:13,047 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:13,047 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:13,047 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:13,047 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:13,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, ASSIGN}] 2023-02-11 03:54:13,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, ASSIGN 2023-02-11 03:54:13,050 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:13,200 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:13,202 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087653201"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087653201"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087653201"}]},"ts":"1676087653201"} 2023-02-11 03:54:13,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:13,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 09e96c0d0642358ef90bd77ad1a2435b, NAME => 'testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:13,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:13,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:13,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,362 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,363 DEBUG [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1 2023-02-11 03:54:13,363 DEBUG [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1 2023-02-11 03:54:13,364 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09e96c0d0642358ef90bd77ad1a2435b columnFamilyName cf1 2023-02-11 03:54:13,365 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] regionserver.HStore(310): Store=09e96c0d0642358ef90bd77ad1a2435b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:13,365 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,366 DEBUG [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2 2023-02-11 03:54:13,366 DEBUG [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2 2023-02-11 03:54:13,368 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09e96c0d0642358ef90bd77ad1a2435b columnFamilyName cf2 2023-02-11 03:54:13,369 INFO [StoreOpener-09e96c0d0642358ef90bd77ad1a2435b-1] regionserver.HStore(310): Store=09e96c0d0642358ef90bd77ad1a2435b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:13,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:13,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:13,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:13,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 09e96c0d0642358ef90bd77ad1a2435b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69690174, jitterRate=0.038464516401290894}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:13,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 09e96c0d0642358ef90bd77ad1a2435b: Running coprocessor pre-open hook at 1676087653361Writing region info on filesystem at 1676087653361Initializing all the Stores at 1676087653361Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087653362 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087653362Cleaning up temporary data from old regions at 1676087653371 (+9 ms)Cleaning up detritus from prior splits at 1676087653371Running coprocessor post-open hooks at 1676087653376 (+5 ms)Region opened successfully at 1676087653376 2023-02-11 03:54:13,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b., pid=27, masterSystemTime=1676087653356 2023-02-11 03:54:13,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,382 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087653381"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087653381"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087653381"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087653381"}]},"ts":"1676087653381"} 2023-02-11 03:54:13,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-11 03:54:13,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 in 180 msec 2023-02-11 03:54:13,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-11 03:54:13,390 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, ASSIGN in 340 msec 2023-02-11 03:54:13,391 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:13,391 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087653391"}]},"ts":"1676087653391"} 2023-02-11 03:54:13,393 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-11 03:54:13,397 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:13,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 394 msec 2023-02-11 03:54:13,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:13,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:13,519 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:13,519 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,520 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:13,520 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:13,520 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,520 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,521 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,521 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,521 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,521 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,522 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,522 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,523 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,523 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,524 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,525 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,526 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,526 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,526 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,526 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:13,526 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:13,526 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,526 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:13,527 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:54:13,527 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,528 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,528 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,528 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,529 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,529 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,538 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/cd7016815aa944d3a108266054fbe194 2023-02-11 03:54:13,559 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/7847594c2c894fad93c75b7c7d28c6e1 2023-02-11 03:54:13,565 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/cd7016815aa944d3a108266054fbe194 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 2023-02-11 03:54:13,571 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:13,571 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/7847594c2c894fad93c75b7c7d28c6e1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 2023-02-11 03:54:13,577 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:13,578 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 51ms, sequenceid=6, compaction requested=false 2023-02-11 03:54:13,578 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-11 03:54:13,579 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:13,579 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,579 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,581 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,581 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,582 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,582 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:13,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12df22b6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:13,584 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:13,584 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,585 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,585 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,585 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,586 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:13,586 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:13,587 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,588 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,588 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,588 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,588 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,588 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,589 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:13,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:13,592 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:13,592 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:13,592 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:13,592 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:13,592 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:13,595 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,595 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,595 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,595 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,595 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,596 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,597 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,597 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:13,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,602 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,603 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:13,603 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,604 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,703 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:13,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:13,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:13,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:13,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:13,724 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:13,724 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,725 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:13,725 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:13,725 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,725 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,727 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,727 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,727 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,727 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:13,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,729 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,729 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,729 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,729 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,730 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,729 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,730 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,731 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:13,732 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:13,732 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,732 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:13,732 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,746 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/5e4194f95d74496ab60dc8a326f47a40 2023-02-11 03:54:13,762 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/b1a9d82cb4864035b57e5ffab44b729a 2023-02-11 03:54:13,768 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/5e4194f95d74496ab60dc8a326f47a40 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 2023-02-11 03:54:13,773 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:13,774 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/b1a9d82cb4864035b57e5ffab44b729a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a 2023-02-11 03:54:13,778 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:13,779 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 47ms, sequenceid=19, compaction requested=false 2023-02-11 03:54:13,779 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:13,780 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,780 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:13,780 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:13,780 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,780 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,780 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,785 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,786 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,786 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,786 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:13,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78ec2e78[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:13,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:13,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:13,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,790 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:13,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:13,791 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,791 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,791 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,796 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,796 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,796 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,797 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,797 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,797 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:13,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:13,801 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:13,801 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:13,801 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:13,801 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:13,801 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,804 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,804 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,805 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,806 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,808 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,808 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,808 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,812 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:13,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:13,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,813 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,814 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,814 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:13,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,814 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,814 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,814 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,832 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 03:54:13,833 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-11 03:54:13,833 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:13,833 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-11 03:54:13,914 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:13,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:13,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:13,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:13,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:13,934 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:13,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:13,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:13,936 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,936 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,938 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,938 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:13,940 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,940 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,940 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,940 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:13,940 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:13,941 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,941 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:13,941 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,941 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,942 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,942 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,942 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,942 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,941 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:13,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,943 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:13,943 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:13,943 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,943 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:13,943 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:13,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,945 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,945 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,945 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,945 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,945 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,954 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/8f701a0e31224a419a663af032402d3a 2023-02-11 03:54:13,967 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/ed26cefede94485e889d05ac1e8f9aff 2023-02-11 03:54:13,973 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/8f701a0e31224a419a663af032402d3a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a 2023-02-11 03:54:13,979 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:13,980 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/ed26cefede94485e889d05ac1e8f9aff as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff 2023-02-11 03:54:13,985 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:13,986 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 43ms, sequenceid=32, compaction requested=true 2023-02-11 03:54:13,986 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:13,986 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:13,986 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:13,986 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:13,986 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,986 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:13,986 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:13,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,988 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,988 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,988 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:13,990 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65ccef3f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:13,990 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:13,990 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,993 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:13,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:13,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:13,993 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:13,995 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,995 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,995 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,996 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,996 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,996 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:13,996 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:13,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:13,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:14,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:14,000 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:14,001 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:14,001 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:14,001 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:14,001 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:14,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,003 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,004 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,004 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,005 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,007 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,007 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,007 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,017 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:14,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,017 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,018 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,018 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:14,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,018 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:14,018 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,119 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:14,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:14,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:14,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:14,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:14,139 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:14,139 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,140 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:14,140 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:14,141 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,141 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,142 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,142 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,142 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,142 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,143 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,146 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,146 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,146 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,146 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,146 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,146 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,146 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,146 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:14,147 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,147 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,147 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,147 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:14,149 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,149 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,149 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,149 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,154 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,161 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/8c7bb75ded9147b6b3436d2de227756e 2023-02-11 03:54:14,174 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/fb3a83ae3188427db99c62b41d8c7954 2023-02-11 03:54:14,179 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/8c7bb75ded9147b6b3436d2de227756e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e 2023-02-11 03:54:14,184 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:14,185 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/fb3a83ae3188427db99c62b41d8c7954 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 2023-02-11 03:54:14,190 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:14,191 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 42ms, sequenceid=45, compaction requested=true 2023-02-11 03:54:14,191 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:14,191 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,191 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:14,191 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:14,191 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,191 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,191 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,193 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,193 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,193 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:14,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@372b2609[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:14,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:14,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,196 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,196 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:14,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,197 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,197 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:14,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,197 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:14,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,201 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,201 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,201 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,201 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:14,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:14,206 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:14,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:14,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:14,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:14,206 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,211 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,212 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,212 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:14,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:14,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,217 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,317 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:14,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:14,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:14,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:14,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:14,340 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:14,340 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,340 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:14,340 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:14,340 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,341 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,342 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,342 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,342 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,343 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:14,344 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,344 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,344 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,345 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,345 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:14,346 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,347 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,347 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,347 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,347 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,347 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,360 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/99c9a5213b8949338fc175e2871fe26b 2023-02-11 03:54:14,374 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/39e2dc47764547969ba01026821eb75c 2023-02-11 03:54:14,380 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/99c9a5213b8949338fc175e2871fe26b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b 2023-02-11 03:54:14,386 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:14,387 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/39e2dc47764547969ba01026821eb75c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c 2023-02-11 03:54:14,393 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:14,394 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 49ms, sequenceid=58, compaction requested=true 2023-02-11 03:54:14,394 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:14,394 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,394 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,398 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,399 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,399 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c0a78c8[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:14,401 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:14,401 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,403 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,403 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,403 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,404 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,403 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:14,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,404 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:14,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:14,405 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,405 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,405 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,406 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,406 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:14,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:14,411 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:14,411 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:14,411 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:14,411 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:14,411 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,414 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,415 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,415 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,417 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,417 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,417 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:14,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:14,423 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,422 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:14,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,423 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,424 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:14,524 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:14,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:14,547 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:14,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:14,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-11 03:54:14,549 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-11 03:54:14,549 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,549 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-11 03:54:14,549 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-11 03:54:14,550 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,550 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,551 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,551 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,551 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,553 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-11 03:54:14,553 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,553 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,553 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,554 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-11 03:54:14,554 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-11 03:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-11 03:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-11 03:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:14,554 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:14,554 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,554 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. started... 2023-02-11 03:54:14,554 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:14,555 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,557 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,557 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,568 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/0b7ce4eeeecd47148ec803de33c84962 2023-02-11 03:54:14,582 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/c4da27264da1437681a2f125a06c3b04 2023-02-11 03:54:14,587 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/0b7ce4eeeecd47148ec803de33c84962 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 2023-02-11 03:54:14,593 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:14,594 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/c4da27264da1437681a2f125a06c3b04 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 2023-02-11 03:54:14,599 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:14,600 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 46ms, sequenceid=71, compaction requested=true 2023-02-11 03:54:14,600 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e96c0d0642358ef90bd77ad1a2435b: 2023-02-11 03:54:14,600 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-11 03:54:14,600 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-11 03:54:14,602 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,602 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,602 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-11 03:54:14,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@458c12df[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:14,604 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-11 03:54:14,604 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,605 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,605 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:14,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,605 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,606 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-11 03:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-11 03:54:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-11 03:54:14,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:54:14,613 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-11 03:54:14,613 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-11 03:54:14,613 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:14,613 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:14,613 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-11 03:54:14,613 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,615 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,614 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,615 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,615 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,617 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,617 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-11 03:54:14,618 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-11 03:54:14,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,620 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,620 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:14,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,621 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,622 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,622 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-11 03:54:14,622 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-11 03:54:14,721 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:14,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:14,985 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 03:54:14,986 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:14,987 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:14,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:14,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:14,988 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:14,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:14,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:14,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:14,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:15,238 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,239 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:15,239 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:15,239 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:15,241 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-11 03:54:15,242 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:15,242 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:15,242 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:15,242 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:15,243 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:15,247 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 03:54:15,249 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:15,250 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:15,250 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:15,250 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:15,251 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,251 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:15,252 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,252 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,253 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:15,254 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,254 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:15,254 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,254 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,254 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,255 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-11 03:54:15,255 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:15,256 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:15,257 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087685257 2023-02-11 03:54:15,259 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:15,259 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,263 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,263 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,263 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:15,263 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087715263 2023-02-11 03:54:15,263 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087775263 2023-02-11 03:54:15,264 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 03:54:15,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:15,266 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:15,266 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:15,267 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:15,514 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,514 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:15,514 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:15,514 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:15,514 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:15,515 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:15,515 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:15,515 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:15,515 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:15,516 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:15,517 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 03:54:15,518 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:15,519 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:15,519 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:15,519 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:15,520 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,520 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:15,520 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,520 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,521 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:15,521 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,521 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:15,521 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,521 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,521 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,522 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:15,522 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:15,523 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:15,523 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087685523 2023-02-11 03:54:15,523 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:15,525 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,527 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,528 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,528 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:15,528 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087715528 2023-02-11 03:54:15,528 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087775528 2023-02-11 03:54:15,528 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:15,528 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:15,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN}] 2023-02-11 03:54:15,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:15,532 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:15,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:15,778 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,778 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:15,779 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:15,779 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:15,780 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:15,782 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 03:54:15,782 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:15,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:15,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:15,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:15,784 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:15,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:15,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:15,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,786 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:15,787 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,787 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,787 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,787 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:54:15,788 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:15,788 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:15,788 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087685788 2023-02-11 03:54:15,790 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:15,790 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:15,793 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,793 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:15,793 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:15,793 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087715793 2023-02-11 03:54:15,793 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087775793 2023-02-11 03:54:15,794 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:15,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN}] 2023-02-11 03:54:15,794 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:15,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:15,797 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:15,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087655797"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087655797"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087655797"}]},"ts":"1676087655797"} 2023-02-11 03:54:15,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:15,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:15,799 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:15,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:15,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:16,044 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,044 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:16,044 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:16,044 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:16,045 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:16,045 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:16,045 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:16,045 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:16,045 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:16,046 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:16,047 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:16,048 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:16,048 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:16,048 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:16,049 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:16,049 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:16,049 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,050 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:16,050 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,050 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,050 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:16,051 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,052 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,052 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,052 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,052 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,052 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,052 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,052 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,052 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:16,053 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:16,053 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:16,053 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b to restore RIT 2023-02-11 03:54:16,054 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087686054 2023-02-11 03:54:16,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,055 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:16,057 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,057 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,058 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:16,058 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087716058 2023-02-11 03:54:16,058 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087776058 2023-02-11 03:54:16,058 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:16,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087656058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087656058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087656058"}]},"ts":"1676087656058"} 2023-02-11 03:54:16,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:16,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:16,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 09e96c0d0642358ef90bd77ad1a2435b, disabling compactions & flushes 2023-02-11 03:54:16,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:16,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:16,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. after waiting 0 ms 2023-02-11 03:54:16,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:16,214 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 09e96c0d0642358ef90bd77ad1a2435b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:54:16,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/45337dfe414947ec94abe8b33d662180 2023-02-11 03:54:16,247 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/bf540f5d05ae4b83986e7381bf71d39a 2023-02-11 03:54:16,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf1/45337dfe414947ec94abe8b33d662180 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 2023-02-11 03:54:16,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:16,259 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/.tmp/cf2/bf540f5d05ae4b83986e7381bf71d39a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a 2023-02-11 03:54:16,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:16,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 51ms, sequenceid=82, compaction requested=true 2023-02-11 03:54:16,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:54:16,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:16,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. 2023-02-11 03:54:16,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 09e96c0d0642358ef90bd77ad1a2435b: Waiting for close lock at 1676087656214Running coprocessor pre-close hooks at 1676087656214Disabling compacts and flushes for region at 1676087656214Disabling writes for close at 1676087656214Obtaining lock to block concurrent updates at 1676087656214Preparing flush snapshotting stores in 09e96c0d0642358ef90bd77ad1a2435b at 1676087656214Finished memstore snapshotting testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087656214Flushing stores of testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. at 1676087656215 (+1 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf1: creating writer at 1676087656216 (+1 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf1: appending metadata at 1676087656218 (+2 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf1: closing flushed file at 1676087656218Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf2: creating writer at 1676087656232 (+14 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf2: appending metadata at 1676087656234 (+2 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf2: closing flushed file at 1676087656234Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf1: reopening flushed file at 1676087656254 (+20 ms)Flushing 09e96c0d0642358ef90bd77ad1a2435b/cf2: reopening flushed file at 1676087656259 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 09e96c0d0642358ef90bd77ad1a2435b in 51ms, sequenceid=82, compaction requested=true at 1676087656265 (+6 ms)Writing region close event to WAL at 1676087656281 (+16 ms)Running coprocessor post-close hooks at 1676087656286 (+5 ms)Closed at 1676087656286 2023-02-11 03:54:16,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED 2023-02-11 03:54:16,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087656289"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087656289"}]},"ts":"1676087656289"} 2023-02-11 03:54:16,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:16,292 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:16,292 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:16,308 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,308 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:16,308 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:16,309 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:16,309 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:16,310 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:16,310 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:16,310 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:16,310 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:16,311 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:16,313 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:16,313 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 03:54:16,313 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 03:54:16,313 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:16,314 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:16,314 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:16,314 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:16,315 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,315 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:16,316 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,316 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,317 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:16,318 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,318 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,318 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,318 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,318 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,318 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,318 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,318 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,318 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,319 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,319 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:54:16,319 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:16,320 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:16,320 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b to restore RIT 2023-02-11 03:54:16,324 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087686324 2023-02-11 03:54:16,325 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,325 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:16,328 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,329 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,329 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:16,329 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087716329 2023-02-11 03:54:16,329 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087776329 2023-02-11 03:54:16,329 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:16,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087656329"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087656329"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087656329"}]},"ts":"1676087656329"} 2023-02-11 03:54:16,332 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:16,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-11 03:54:16,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 in 272 msec 2023-02-11 03:54:16,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:16,336 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:16,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:16,579 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:16,580 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:16,580 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:16,581 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:16,582 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:16,583 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 03:54:16,583 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:16,583 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:16,584 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:16,584 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:16,584 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:16,584 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,585 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:16,585 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,585 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,586 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:16,586 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,586 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,587 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,587 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:16,587 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,587 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,587 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,587 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:16,587 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:16,587 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:16,588 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:16,588 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:16,588 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b to restore RIT 2023-02-11 03:54:16,589 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087686589 2023-02-11 03:54:16,590 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:16,590 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,592 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,592 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,592 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:16,592 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087716592 2023-02-11 03:54:16,592 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087776592 2023-02-11 03:54:16,593 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:16,593 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,594 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:16,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-11 03:54:16,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN in 800 msec 2023-02-11 03:54:16,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:16,601 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:16,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:16,843 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,843 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:16,843 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:16,843 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:16,843 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:16,843 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:16,844 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:16,844 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:16,844 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:16,845 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:16,846 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:16,846 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:16,846 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:16,847 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:16,847 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:16,847 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:16,848 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:16,848 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,848 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:16,849 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,849 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:16,849 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:16,850 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:16,850 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:16,851 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:16,851 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:16,852 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087686852 2023-02-11 03:54:16,853 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:16,853 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,856 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,856 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:16,856 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:16,856 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087716856 2023-02-11 03:54:16,856 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087776856 2023-02-11 03:54:16,856 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:16,856 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,856 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,857 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:16,861 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:16,871 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=09e96c0d0642358ef90bd77ad1a2435b, threads=14 2023-02-11 03:54:16,872 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,872 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,872 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,872 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,872 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,873 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,880 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,880 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,880 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,881 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,881 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,882 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,882 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,882 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,994 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,995 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:16,996 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,013 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,014 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,016 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,016 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,020 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,020 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,024 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,028 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,032 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,033 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,034 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,034 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 09e96c0d0642358ef90bd77ad1a2435b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:17,068 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:17,068 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:17,068 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:17,106 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:17,107 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:17,107 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:17,108 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:17,108 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:17,110 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 03:54:17,110 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:17,110 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:17,110 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:17,111 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:17,111 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:17,111 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:17,112 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,112 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:17,113 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,113 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,114 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:17,114 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,115 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:17,115 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:17,115 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:17,115 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,115 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,115 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,115 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,115 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,115 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:54:17,116 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:17,116 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:17,117 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087687117 2023-02-11 03:54:17,118 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:17,118 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,120 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,121 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,121 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:17,121 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087717121 2023-02-11 03:54:17,121 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087777121 2023-02-11 03:54:17,121 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:17,130 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=09e96c0d0642358ef90bd77ad1a2435b, threads=14 2023-02-11 03:54:17,131 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,131 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,131 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,132 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,132 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,132 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,132 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,133 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,134 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,328 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,329 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,356 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,357 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,362 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,364 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,373 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,387 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,387 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,388 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,392 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,402 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,402 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,404 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a for region: 09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:17,404 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 09e96c0d0642358ef90bd77ad1a2435b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:17,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:17,450 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:17,452 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:17,452 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:17,452 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:17,452 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:17,621 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,621 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:17,621 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:17,621 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:17,622 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:17,622 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:17,622 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:17,622 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:17,622 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:17,623 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:17,624 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 03:54:17,624 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:17,624 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:17,624 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:17,624 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:17,625 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:17,625 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:17,625 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,625 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:17,626 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,626 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,626 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:17,626 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,627 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:17,627 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:17,627 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:17,627 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,627 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,627 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,627 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,627 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,627 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:17,628 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:17,628 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:17,629 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087687629 2023-02-11 03:54:17,630 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:17,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,633 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,633 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,633 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:17,633 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087717633 2023-02-11 03:54:17,633 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087777633 2023-02-11 03:54:17,633 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:17,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:17,639 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:17,639 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:17,639 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:17,792 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:17,884 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:17,884 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:17,884 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:17,885 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:17,886 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 03:54:17,886 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:17,886 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:17,886 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:17,887 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:17,887 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:17,887 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:17,887 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,888 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:17,888 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,888 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:17,888 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:17,889 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,889 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,889 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=09e96c0d0642358ef90bd77ad1a2435b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,889 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 09e96c0d0642358ef90bd77ad1a2435b, server=jenkins-hbase4.apache.org,33925,1676087644887 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:17,890 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:17,890 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:17,891 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:17,891 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087687891 2023-02-11 03:54:17,891 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:17,892 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:17,894 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,895 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=09e96c0d0642358ef90bd77ad1a2435b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:17,895 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:17,895 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087717895 2023-02-11 03:54:17,895 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087777895 2023-02-11 03:54:17,895 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:17,898 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087657898"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676087657898"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676087657898"}]},"ts":"1676087657898"} 2023-02-11 03:54:17,898 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087657898"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087657898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087657898"}]},"ts":"1676087657898"} 2023-02-11 03:54:17,898 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087657898"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087657898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087657898"}]},"ts":"1676087657898"} 2023-02-11 03:54:17,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, ASSIGN}] 2023-02-11 03:54:17,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, ASSIGN 2023-02-11 03:54:17,905 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:17,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, ASSIGN 2023-02-11 03:54:17,907 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:54:18,055 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:54:18,057 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c21eac229581dfd2abc660b4eaeee1de, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,057 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658057"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087658057"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087658057"}]},"ts":"1676087658057"} 2023-02-11 03:54:18,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure c21eac229581dfd2abc660b4eaeee1de, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:18,062 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=e400ebe998312232ebe6c6df0af8b6ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,063 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658062"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087658062"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087658062"}]},"ts":"1676087658062"} 2023-02-11 03:54:18,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure e400ebe998312232ebe6c6df0af8b6ed, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:18,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e400ebe998312232ebe6c6df0af8b6ed, NAME => 'testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 03:54:18,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,216 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,217 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1 2023-02-11 03:54:18,217 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1 2023-02-11 03:54:18,218 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e400ebe998312232ebe6c6df0af8b6ed columnFamilyName cf1 2023-02-11 03:54:18,232 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-top 2023-02-11 03:54:18,236 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-top 2023-02-11 03:54:18,240 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-top 2023-02-11 03:54:18,245 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-top 2023-02-11 03:54:18,250 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-top 2023-02-11 03:54:18,254 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-top 2023-02-11 03:54:18,259 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-top 2023-02-11 03:54:18,259 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(310): Store=e400ebe998312232ebe6c6df0af8b6ed/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:18,259 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,260 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2 2023-02-11 03:54:18,260 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2 2023-02-11 03:54:18,260 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e400ebe998312232ebe6c6df0af8b6ed columnFamilyName cf2 2023-02-11 03:54:18,275 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-top 2023-02-11 03:54:18,280 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-top 2023-02-11 03:54:18,284 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-top 2023-02-11 03:54:18,288 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-top 2023-02-11 03:54:18,292 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-top 2023-02-11 03:54:18,295 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-top 2023-02-11 03:54:18,300 DEBUG [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-top 2023-02-11 03:54:18,300 INFO [StoreOpener-e400ebe998312232ebe6c6df0af8b6ed-1] regionserver.HStore(310): Store=e400ebe998312232ebe6c6df0af8b6ed/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:18,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:18,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e400ebe998312232ebe6c6df0af8b6ed; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67085014, jitterRate=-3.553926944732666E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:18,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e400ebe998312232ebe6c6df0af8b6ed: Running coprocessor pre-open hook at 1676087658215Writing region info on filesystem at 1676087658215Initializing all the Stores at 1676087658216 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087658216Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087658216Cleaning up temporary data from old regions at 1676087658302 (+86 ms)Cleaning up detritus from prior splits at 1676087658303 (+1 ms)Running coprocessor post-open hooks at 1676087658306 (+3 ms)Region opened successfully at 1676087658306 2023-02-11 03:54:18,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed., pid=34, masterSystemTime=1676087658211 2023-02-11 03:54:18,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:18,308 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:18,308 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,308 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): e400ebe998312232ebe6c6df0af8b6ed/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e400ebe998312232ebe6c6df0af8b6ed/cf1 in testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,308 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): e400ebe998312232ebe6c6df0af8b6ed/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e400ebe998312232ebe6c6df0af8b6ed/cf2 in testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/.tmp, totalSize=34.9 K 2023-02-11 03:54:18,308 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/.tmp, totalSize=34.9 K 2023-02-11 03:54:18,309 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087653509 2023-02-11 03:54:18,309 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087653509 2023-02-11 03:54:18,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,310 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087653705 2023-02-11 03:54:18,310 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087653705 2023-02-11 03:54:18,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c21eac229581dfd2abc660b4eaeee1de, NAME => 'testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 03:54:18,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=e400ebe998312232ebe6c6df0af8b6ed, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,310 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087653916 2023-02-11 03:54:18,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:18,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658310"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087658310"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087658310"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087658310"}]},"ts":"1676087658310"} 2023-02-11 03:54:18,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,311 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087653916 2023-02-11 03:54:18,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:18,311 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087654121 2023-02-11 03:54:18,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,311 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087654121 2023-02-11 03:54:18,312 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087654319 2023-02-11 03:54:18,312 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087654319 2023-02-11 03:54:18,312 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087654527 2023-02-11 03:54:18,313 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087654527 2023-02-11 03:54:18,313 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,313 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087654723 2023-02-11 03:54:18,313 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087654723 2023-02-11 03:54:18,315 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1 2023-02-11 03:54:18,315 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1 2023-02-11 03:54:18,316 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c21eac229581dfd2abc660b4eaeee1de columnFamilyName cf1 2023-02-11 03:54:18,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-11 03:54:18,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure e400ebe998312232ebe6c6df0af8b6ed, server=jenkins-hbase4.apache.org,33925,1676087644887 in 248 msec 2023-02-11 03:54:18,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, ASSIGN in 414 msec 2023-02-11 03:54:18,333 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): e400ebe998312232ebe6c6df0af8b6ed#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:18,336 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): e400ebe998312232ebe6c6df0af8b6ed#cf2#compaction#33 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:18,338 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-bottom 2023-02-11 03:54:18,343 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-bottom 2023-02-11 03:54:18,347 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-bottom 2023-02-11 03:54:18,354 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-bottom 2023-02-11 03:54:18,355 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/.tmp/cf1/2aae31f89ac846cab0ab7d1c7d86f14e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/2aae31f89ac846cab0ab7d1c7d86f14e 2023-02-11 03:54:18,361 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-bottom 2023-02-11 03:54:18,368 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-bottom 2023-02-11 03:54:18,370 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e400ebe998312232ebe6c6df0af8b6ed/cf1 of e400ebe998312232ebe6c6df0af8b6ed into 2aae31f89ac846cab0ab7d1c7d86f14e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:18,370 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e400ebe998312232ebe6c6df0af8b6ed: 2023-02-11 03:54:18,370 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed., storeName=e400ebe998312232ebe6c6df0af8b6ed/cf1, priority=9, startTime=1676087658307; duration=0sec 2023-02-11 03:54:18,370 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,371 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/.tmp/cf2/cb693d4c676144f29006cb41b3ca3cee as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/cb693d4c676144f29006cb41b3ca3cee 2023-02-11 03:54:18,373 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-bottom 2023-02-11 03:54:18,373 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(310): Store=c21eac229581dfd2abc660b4eaeee1de/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:18,374 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,374 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2 2023-02-11 03:54:18,375 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2 2023-02-11 03:54:18,375 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c21eac229581dfd2abc660b4eaeee1de columnFamilyName cf2 2023-02-11 03:54:18,378 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e400ebe998312232ebe6c6df0af8b6ed/cf2 of e400ebe998312232ebe6c6df0af8b6ed into cb693d4c676144f29006cb41b3ca3cee(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:18,378 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e400ebe998312232ebe6c6df0af8b6ed: 2023-02-11 03:54:18,378 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed., storeName=e400ebe998312232ebe6c6df0af8b6ed/cf2, priority=9, startTime=1676087658308; duration=0sec 2023-02-11 03:54:18,379 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,389 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-bottom 2023-02-11 03:54:18,393 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-bottom 2023-02-11 03:54:18,398 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-bottom 2023-02-11 03:54:18,404 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-bottom 2023-02-11 03:54:18,408 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-bottom 2023-02-11 03:54:18,412 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-bottom 2023-02-11 03:54:18,417 DEBUG [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-bottom 2023-02-11 03:54:18,417 INFO [StoreOpener-c21eac229581dfd2abc660b4eaeee1de-1] regionserver.HStore(310): Store=c21eac229581dfd2abc660b4eaeee1de/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:18,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:18,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c21eac229581dfd2abc660b4eaeee1de; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67520013, jitterRate=0.00612659752368927}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:18,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c21eac229581dfd2abc660b4eaeee1de: Running coprocessor pre-open hook at 1676087658311Writing region info on filesystem at 1676087658311Initializing all the Stores at 1676087658312 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087658312Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087658313 (+1 ms)Cleaning up temporary data from old regions at 1676087658419 (+106 ms)Cleaning up detritus from prior splits at 1676087658420 (+1 ms)Running coprocessor post-open hooks at 1676087658423 (+3 ms)Region opened successfully at 1676087658423 2023-02-11 03:54:18,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de., pid=33, masterSystemTime=1676087658211 2023-02-11 03:54:18,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,424 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:18,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:18,425 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,425 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:18,425 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): c21eac229581dfd2abc660b4eaeee1de/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:18,425 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,425 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c21eac229581dfd2abc660b4eaeee1de/cf1 in testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,425 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): c21eac229581dfd2abc660b4eaeee1de/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:18,425 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c21eac229581dfd2abc660b4eaeee1de/cf2 in testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,425 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/.tmp, totalSize=34.9 K 2023-02-11 03:54:18,426 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/.tmp, totalSize=34.9 K 2023-02-11 03:54:18,426 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087653509 2023-02-11 03:54:18,426 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087653509 2023-02-11 03:54:18,426 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087653705 2023-02-11 03:54:18,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,427 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087653705 2023-02-11 03:54:18,427 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087653916 2023-02-11 03:54:18,427 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=c21eac229581dfd2abc660b4eaeee1de, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,427 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658427"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087658427"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087658427"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087658427"}]},"ts":"1676087658427"} 2023-02-11 03:54:18,428 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087653916 2023-02-11 03:54:18,428 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087654121 2023-02-11 03:54:18,428 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087654121 2023-02-11 03:54:18,429 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087654319 2023-02-11 03:54:18,429 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087654319 2023-02-11 03:54:18,429 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087654527 2023-02-11 03:54:18,430 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087654527 2023-02-11 03:54:18,430 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087654723 2023-02-11 03:54:18,430 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087654723 2023-02-11 03:54:18,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-11 03:54:18,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure c21eac229581dfd2abc660b4eaeee1de, server=jenkins-hbase4.apache.org,33925,1676087644887 in 373 msec 2023-02-11 03:54:18,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-11 03:54:18,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, ASSIGN in 532 msec 2023-02-11 03:54:18,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed in 3.4510 sec 2023-02-11 03:54:18,452 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): c21eac229581dfd2abc660b4eaeee1de#cf2#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:18,452 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): c21eac229581dfd2abc660b4eaeee1de#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:18,474 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/.tmp/cf2/616354853b29411496cd081d10ca6b50 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/616354853b29411496cd081d10ca6b50 2023-02-11 03:54:18,476 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/.tmp/cf1/c52c948270e443d9a3921c5eb032d9ec as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/c52c948270e443d9a3921c5eb032d9ec 2023-02-11 03:54:18,486 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c21eac229581dfd2abc660b4eaeee1de/cf1 of c21eac229581dfd2abc660b4eaeee1de into c52c948270e443d9a3921c5eb032d9ec(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:18,486 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c21eac229581dfd2abc660b4eaeee1de/cf2 of c21eac229581dfd2abc660b4eaeee1de into 616354853b29411496cd081d10ca6b50(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:18,486 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c21eac229581dfd2abc660b4eaeee1de: 2023-02-11 03:54:18,486 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c21eac229581dfd2abc660b4eaeee1de: 2023-02-11 03:54:18,486 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de., storeName=c21eac229581dfd2abc660b4eaeee1de/cf1, priority=9, startTime=1676087658423; duration=0sec 2023-02-11 03:54:18,486 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de., storeName=c21eac229581dfd2abc660b4eaeee1de/cf2, priority=9, startTime=1676087658424; duration=0sec 2023-02-11 03:54:18,487 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,487 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:18,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33925] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:55758 deadline: 1676087718646, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b. is not online on jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,759 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:54:18,766 INFO [Listener at localhost/45015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 03:54:18,768 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:54:18,784 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:18,785 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-11 03:54:18,786 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-11 03:54:18,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:18,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:18,790 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087658790"}]},"ts":"1676087658790"} 2023-02-11 03:54:18,791 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-11 03:54:18,793 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-11 03:54:18,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, UNASSIGN}] 2023-02-11 03:54:18,795 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, UNASSIGN 2023-02-11 03:54:18,796 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=e400ebe998312232ebe6c6df0af8b6ed, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,796 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658795"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087658795"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087658795"}]},"ts":"1676087658795"} 2023-02-11 03:54:18,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure e400ebe998312232ebe6c6df0af8b6ed, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:18,800 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, UNASSIGN 2023-02-11 03:54:18,800 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c21eac229581dfd2abc660b4eaeee1de, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:18,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658800"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087658800"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087658800"}]},"ts":"1676087658800"} 2023-02-11 03:54:18,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure c21eac229581dfd2abc660b4eaeee1de, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:18,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:18,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c21eac229581dfd2abc660b4eaeee1de, disabling compactions & flushes 2023-02-11 03:54:18,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. after waiting 0 ms 2023-02-11 03:54:18,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,960 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-bottom] to archive 2023-02-11 03:54:18,961 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:18,963 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,964 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,965 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,967 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,968 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,969 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,970 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,972 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-bottom] to archive 2023-02-11 03:54:18,973 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:18,974 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,976 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,977 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,978 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,979 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,980 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,982 DEBUG [StoreCloser-testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,985 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:54:18,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:18,986 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de. 2023-02-11 03:54:18,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c21eac229581dfd2abc660b4eaeee1de: Waiting for close lock at 1676087658949Running coprocessor pre-close hooks at 1676087658949Disabling compacts and flushes for region at 1676087658949Disabling writes for close at 1676087658949Writing region close event to WAL at 1676087658983 (+34 ms)Running coprocessor post-close hooks at 1676087658986 (+3 ms)Closed at 1676087658986 2023-02-11 03:54:18,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:18,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:18,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e400ebe998312232ebe6c6df0af8b6ed, disabling compactions & flushes 2023-02-11 03:54:18,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. after waiting 0 ms 2023-02-11 03:54:18,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:18,988 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=c21eac229581dfd2abc660b4eaeee1de, regionState=CLOSED 2023-02-11 03:54:18,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087658988"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087658988"}]},"ts":"1676087658988"} 2023-02-11 03:54:18,989 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180-top] to archive 2023-02-11 03:54:18,990 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:18,992 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-11 03:54:18,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure c21eac229581dfd2abc660b4eaeee1de, server=jenkins-hbase4.apache.org,33925,1676087644887 in 188 msec 2023-02-11 03:54:18,993 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=c21eac229581dfd2abc660b4eaeee1de, UNASSIGN in 199 msec 2023-02-11 03:54:18,995 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,997 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:18,999 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,000 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,001 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,003 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a-top] to archive 2023-02-11 03:54:19,004 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:19,005 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,006 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,008 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,009 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,010 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,011 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,012 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:54:19,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:19,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed. 2023-02-11 03:54:19,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e400ebe998312232ebe6c6df0af8b6ed: Waiting for close lock at 1676087658988Running coprocessor pre-close hooks at 1676087658988Disabling compacts and flushes for region at 1676087658988Disabling writes for close at 1676087658988Writing region close event to WAL at 1676087659013 (+25 ms)Running coprocessor post-close hooks at 1676087659016 (+3 ms)Closed at 1676087659016 2023-02-11 03:54:19,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:19,018 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=e400ebe998312232ebe6c6df0af8b6ed, regionState=CLOSED 2023-02-11 03:54:19,018 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087659018"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087659018"}]},"ts":"1676087659018"} 2023-02-11 03:54:19,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-11 03:54:19,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure e400ebe998312232ebe6c6df0af8b6ed, server=jenkins-hbase4.apache.org,33925,1676087644887 in 223 msec 2023-02-11 03:54:19,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-11 03:54:19,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=e400ebe998312232ebe6c6df0af8b6ed, UNASSIGN in 228 msec 2023-02-11 03:54:19,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087659024"}]},"ts":"1676087659024"} 2023-02-11 03:54:19,025 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-11 03:54:19,028 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-11 03:54:19,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 243 msec 2023-02-11 03:54:19,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:19,092 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-11 03:54:19,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-11 03:54:19,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,095 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,096 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 03:54:19,102 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:19,102 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:19,104 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/recovered.edits] 2023-02-11 03:54:19,105 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/recovered.edits] 2023-02-11 03:54:19,105 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/recovered.edits] 2023-02-11 03:54:19,108 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/2aae31f89ac846cab0ab7d1c7d86f14e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf1/2aae31f89ac846cab0ab7d1c7d86f14e 2023-02-11 03:54:19,109 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/0b7ce4eeeecd47148ec803de33c84962 2023-02-11 03:54:19,109 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/.regioninfo to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/.regioninfo 2023-02-11 03:54:19,111 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/45337dfe414947ec94abe8b33d662180 2023-02-11 03:54:19,112 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/cb693d4c676144f29006cb41b3ca3cee to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/cf2/cb693d4c676144f29006cb41b3ca3cee 2023-02-11 03:54:19,113 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/5e4194f95d74496ab60dc8a326f47a40 2023-02-11 03:54:19,113 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,114 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8c7bb75ded9147b6b3436d2de227756e 2023-02-11 03:54:19,115 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,115 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/.regioninfo to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/.regioninfo 2023-02-11 03:54:19,116 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/8f701a0e31224a419a663af032402d3a 2023-02-11 03:54:19,117 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,118 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/99c9a5213b8949338fc175e2871fe26b 2023-02-11 03:54:19,119 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,119 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/0b7ce4eeeecd47148ec803de33c84962.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,119 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf1/cd7016815aa944d3a108266054fbe194 2023-02-11 03:54:19,120 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,121 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/45337dfe414947ec94abe8b33d662180.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,122 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,123 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/5e4194f95d74496ab60dc8a326f47a40.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,124 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/39e2dc47764547969ba01026821eb75c 2023-02-11 03:54:19,124 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,125 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/8c7bb75ded9147b6b3436d2de227756e.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,125 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/7847594c2c894fad93c75b7c7d28c6e1 2023-02-11 03:54:19,127 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/8f701a0e31224a419a663af032402d3a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,127 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/b1a9d82cb4864035b57e5ffab44b729a 2023-02-11 03:54:19,128 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/99c9a5213b8949338fc175e2871fe26b.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,129 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,129 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/bf540f5d05ae4b83986e7381bf71d39a 2023-02-11 03:54:19,130 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf1/cd7016815aa944d3a108266054fbe194.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,130 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,130 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/c4da27264da1437681a2f125a06c3b04 2023-02-11 03:54:19,132 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,132 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/ed26cefede94485e889d05ac1e8f9aff 2023-02-11 03:54:19,134 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/cf2/fb3a83ae3188427db99c62b41d8c7954 2023-02-11 03:54:19,134 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,134 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/39e2dc47764547969ba01026821eb75c.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,137 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,137 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/7847594c2c894fad93c75b7c7d28c6e1.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,140 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b/recovered.edits/85.seqid 2023-02-11 03:54:19,140 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/b1a9d82cb4864035b57e5ffab44b729a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,141 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,141 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,142 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/c21eac229581dfd2abc660b4eaeee1de/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,142 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/bf540f5d05ae4b83986e7381bf71d39a.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,144 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/c4da27264da1437681a2f125a06c3b04.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,145 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/c52c948270e443d9a3921c5eb032d9ec to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf1/c52c948270e443d9a3921c5eb032d9ec 2023-02-11 03:54:19,145 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/ed26cefede94485e889d05ac1e8f9aff.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,146 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/e400ebe998312232ebe6c6df0af8b6ed/cf2/fb3a83ae3188427db99c62b41d8c7954.09e96c0d0642358ef90bd77ad1a2435b 2023-02-11 03:54:19,147 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/616354853b29411496cd081d10ca6b50 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/cf2/616354853b29411496cd081d10ca6b50 2023-02-11 03:54:19,149 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed/recovered.edits/90.seqid 2023-02-11 03:54:19,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de/recovered.edits/90.seqid 2023-02-11 03:54:19,151 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:19,151 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitWithoutPONR/c21eac229581dfd2abc660b4eaeee1de 2023-02-11 03:54:19,151 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-11 03:54:19,154 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,156 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-11 03:54:19,158 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-11 03:54:19,160 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,160 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-11 03:54:19,160 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087659160"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:19,160 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087659160"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:19,160 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087659160"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:19,162 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:54:19,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 09e96c0d0642358ef90bd77ad1a2435b, NAME => 'testSplitWithoutPONR,,1676087653003.09e96c0d0642358ef90bd77ad1a2435b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c21eac229581dfd2abc660b4eaeee1de, NAME => 'testSplitWithoutPONR,,1676087654986.c21eac229581dfd2abc660b4eaeee1de.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => e400ebe998312232ebe6c6df0af8b6ed, NAME => 'testSplitWithoutPONR,41,1676087654986.e400ebe998312232ebe6c6df0af8b6ed.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 03:54:19,162 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-11 03:54:19,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087659162"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:19,163 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-11 03:54:19,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:19,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 73 msec 2023-02-11 03:54:19,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-11 03:54:19,197 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-11 03:54:19,212 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=394 (was 389) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33564 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43240 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33338 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:38524 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x440c1d01-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33586 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:45426 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:54114 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43750 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:53550 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:58918 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43442 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33294 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:58652 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43784 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:43532 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-753551362_15 at /127.0.0.1:33230 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=843 (was 847), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 243), ProcessCount=168 (was 168), AvailableMemoryMB=3298 (was 3424) 2023-02-11 03:54:19,227 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=394, OpenFileDescriptor=843, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=168, AvailableMemoryMB=3297 2023-02-11 03:54:19,227 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:19,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:19,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:19,232 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:19,233 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:19,234 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:19,235 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,236 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 empty. 2023-02-11 03:54:19,236 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,237 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 03:54:19,248 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:19,250 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6a2d161464af91aca78ab4c3203d8177, NAME => 'testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 6a2d161464af91aca78ab4c3203d8177, disabling compactions & flushes 2023-02-11 03:54:19,258 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. after waiting 0 ms 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,258 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,258 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 6a2d161464af91aca78ab4c3203d8177: Waiting for close lock at 1676087659258Disabling compacts and flushes for region at 1676087659258Disabling writes for close at 1676087659258Writing region close event to WAL at 1676087659258Closed at 1676087659258 2023-02-11 03:54:19,260 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:19,261 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087659261"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087659261"}]},"ts":"1676087659261"} 2023-02-11 03:54:19,262 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:19,263 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:19,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087659263"}]},"ts":"1676087659263"} 2023-02-11 03:54:19,265 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 03:54:19,269 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:19,270 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:19,270 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:19,270 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:19,270 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:19,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN}] 2023-02-11 03:54:19,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:19,272 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:19,422 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:19,424 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087659424"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087659424"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087659424"}]},"ts":"1676087659424"} 2023-02-11 03:54:19,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:19,579 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,579 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:54:19,582 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49614, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:54:19,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a2d161464af91aca78ab4c3203d8177, NAME => 'testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:19,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,590 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,592 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1 2023-02-11 03:54:19,592 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1 2023-02-11 03:54:19,592 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a2d161464af91aca78ab4c3203d8177 columnFamilyName cf1 2023-02-11 03:54:19,593 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(310): Store=6a2d161464af91aca78ab4c3203d8177/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:19,593 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,594 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2 2023-02-11 03:54:19,594 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2 2023-02-11 03:54:19,595 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a2d161464af91aca78ab4c3203d8177 columnFamilyName cf2 2023-02-11 03:54:19,595 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(310): Store=6a2d161464af91aca78ab4c3203d8177/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:19,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:19,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:19,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:19,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a2d161464af91aca78ab4c3203d8177; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59771208, jitterRate=-0.10933959484100342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:19,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a2d161464af91aca78ab4c3203d8177: Running coprocessor pre-open hook at 1676087659589Writing region info on filesystem at 1676087659589Initializing all the Stores at 1676087659590 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087659590Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087659590Cleaning up temporary data from old regions at 1676087659597 (+7 ms)Cleaning up detritus from prior splits at 1676087659597Running coprocessor post-open hooks at 1676087659602 (+5 ms)Region opened successfully at 1676087659602 2023-02-11 03:54:19,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177., pid=43, masterSystemTime=1676087659579 2023-02-11 03:54:19,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087659607"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087659607"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087659607"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087659607"}]},"ts":"1676087659607"} 2023-02-11 03:54:19,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-11 03:54:19,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 in 183 msec 2023-02-11 03:54:19,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-11 03:54:19,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN in 341 msec 2023-02-11 03:54:19,613 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:19,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087659614"}]},"ts":"1676087659614"} 2023-02-11 03:54:19,615 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 03:54:19,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:19,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 389 msec 2023-02-11 03:54:19,735 DEBUG [Listener at localhost/45015] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:19,739 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49620, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:19,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:19,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:19,747 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:19,747 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,747 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:19,747 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:19,747 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,747 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,748 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,748 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,748 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,748 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,749 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,749 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,749 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,750 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:19,751 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:19,751 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,751 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:19,752 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:54:19,754 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,754 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,754 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,754 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,755 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,755 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,763 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/f94605cc975f45e1986f1d4d481a2bf3 2023-02-11 03:54:19,777 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/2587059ea4c648fca240f5cc07e382a8 2023-02-11 03:54:19,781 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/f94605cc975f45e1986f1d4d481a2bf3 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 2023-02-11 03:54:19,786 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:19,786 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/2587059ea4c648fca240f5cc07e382a8 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 2023-02-11 03:54:19,790 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:19,791 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 39ms, sequenceid=6, compaction requested=false 2023-02-11 03:54:19,791 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-11 03:54:19,792 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:19,792 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,792 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,793 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,793 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,794 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,794 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:19,796 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:19,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@473ba846[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:19,796 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:19,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,797 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:19,797 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:19,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:19,800 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,801 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,801 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:19,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:19,804 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:19,804 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:19,804 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:19,804 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:19,804 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,806 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,806 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,807 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,807 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,809 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,809 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,810 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,813 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:19,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:19,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,812 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:19,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,813 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:19,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,914 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:19,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:19,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:19,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:19,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:19,933 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:19,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:19,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:19,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,937 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,937 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,937 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,937 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:19,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,940 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:19,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:19,940 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,940 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,940 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:19,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:19,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:19,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,944 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,944 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,944 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,944 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:19,946 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,947 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,956 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/62228d4a95504ab4be8ae96e7377cbf3 2023-02-11 03:54:19,979 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/21f16c1505e5466282edb372ef21bc77 2023-02-11 03:54:19,984 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/62228d4a95504ab4be8ae96e7377cbf3 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 2023-02-11 03:54:19,989 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:19,990 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/21f16c1505e5466282edb372ef21bc77 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 2023-02-11 03:54:19,995 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:19,996 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 52ms, sequenceid=19, compaction requested=false 2023-02-11 03:54:19,996 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:19,996 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:19,996 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:19,998 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,998 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:19,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,998 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:19,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,998 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:19,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:19,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@294ec5c6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:20,000 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:20,000 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,003 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,003 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,004 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:20,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,004 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:20,008 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,008 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,009 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,009 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:20,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,013 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:20,013 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,013 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,013 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:20,013 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,016 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,017 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,018 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,018 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,020 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,020 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,020 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,022 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,023 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:20,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,023 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,024 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,124 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:20,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:20,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:20,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:20,143 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,143 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,143 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:20,143 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:20,144 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,144 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,145 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,145 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,145 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,145 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,146 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,146 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:20,146 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,148 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,148 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,148 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,152 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:20,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,152 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,152 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,152 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:20,152 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,153 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,153 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,153 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,153 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:20,154 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,156 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,156 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,156 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,156 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,156 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,168 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/cb07deae2fcd4fe5af64fe6b950af447 2023-02-11 03:54:20,188 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/a175abc0fc44470ba56639f6fc6ae59b 2023-02-11 03:54:20,193 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/cb07deae2fcd4fe5af64fe6b950af447 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 2023-02-11 03:54:20,197 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:20,198 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/a175abc0fc44470ba56639f6fc6ae59b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b 2023-02-11 03:54:20,202 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:20,203 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 50ms, sequenceid=32, compaction requested=true 2023-02-11 03:54:20,203 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:20,203 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:20,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:20,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,203 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,205 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,205 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,205 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,205 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:20,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@260d241f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:20,208 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:20,208 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:20,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:20,210 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:20,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,210 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:20,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,213 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,214 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,214 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:20,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,217 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:20,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:20,217 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,219 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,220 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,220 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,221 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,221 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,223 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,223 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,223 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,230 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,230 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:20,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,231 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,331 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:20,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:20,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:20,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:20,349 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,349 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,349 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:20,349 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:20,349 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,349 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,350 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,351 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,350 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,351 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,351 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,353 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,353 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,353 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,354 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,355 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,355 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:20,356 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,356 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:20,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,356 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:20,357 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,358 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,358 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,367 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/9b4d2b46c1d84ad089e37360e4e0de1e 2023-02-11 03:54:20,379 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/c082ae972b4e460cb0ca58be7cd7abc4 2023-02-11 03:54:20,383 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/9b4d2b46c1d84ad089e37360e4e0de1e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e 2023-02-11 03:54:20,387 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:20,388 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/c082ae972b4e460cb0ca58be7cd7abc4 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 2023-02-11 03:54:20,392 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:20,393 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 36ms, sequenceid=45, compaction requested=true 2023-02-11 03:54:20,393 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:20,393 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,393 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,395 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,395 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,396 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,396 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:20,397 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b2d0eb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:20,397 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:20,398 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,399 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:20,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:20,399 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,401 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,401 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,402 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:20,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,405 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:20,405 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,405 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,405 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:20,405 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,407 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,408 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,408 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,409 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,410 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,410 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,411 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,413 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:20,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,414 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,415 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,514 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:20,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:20,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:20,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:20,534 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,534 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,534 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:20,534 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:20,535 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,535 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,536 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,536 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,536 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,538 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:20,539 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:20,539 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,539 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:20,539 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,540 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,549 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/319b67b292c344929e2b13bef796764c 2023-02-11 03:54:20,565 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c 2023-02-11 03:54:20,569 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/319b67b292c344929e2b13bef796764c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c 2023-02-11 03:54:20,574 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:20,574 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c 2023-02-11 03:54:20,578 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:20,579 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 40ms, sequenceid=58, compaction requested=true 2023-02-11 03:54:20,579 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:20,579 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,579 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:20,579 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,580 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,582 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,582 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,583 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,583 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:20,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19d12143[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:20,585 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:20,585 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,586 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:20,586 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,587 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:20,590 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,590 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,590 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,590 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,590 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,591 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,591 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,597 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,597 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,597 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:20,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,600 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:20,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:20,600 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,601 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,601 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,601 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,602 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,605 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,605 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,605 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,608 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,608 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:20,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,609 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,709 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:20,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:20,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:20,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-11 03:54:20,728 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,728 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,728 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-11 03:54:20,728 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:54:20,728 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,728 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,729 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,729 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,730 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,730 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:20,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,731 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-11 03:54:20,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:20,732 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,732 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:20,733 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. started... 2023-02-11 03:54:20,733 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:54:20,733 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,733 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,733 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:20,735 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,735 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,735 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,736 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,736 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,736 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,750 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/88f9802016fb451fb1e48baf455660d9 2023-02-11 03:54:20,766 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/e04e081a3651417abde5953b14c8c0fa 2023-02-11 03:54:20,772 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/88f9802016fb451fb1e48baf455660d9 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 2023-02-11 03:54:20,777 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:20,778 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/e04e081a3651417abde5953b14c8c0fa as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa 2023-02-11 03:54:20,782 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:20,783 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 50ms, sequenceid=71, compaction requested=true 2023-02-11 03:54:20,783 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:20,783 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:20,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:20,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:20,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-11 03:54:20,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-11 03:54:20,785 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,785 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:20,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-11 03:54:20,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@466ea74a[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:20,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:54:20,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,789 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,790 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:20,790 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:20,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-11 03:54:20,790 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:20,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,797 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,798 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-11 03:54:20,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:20,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,802 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-11 03:54:20,802 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,802 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,802 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-11 03:54:20,802 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,803 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,804 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,806 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,806 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,806 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-11 03:54:20,808 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,808 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,809 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,815 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-11 03:54:20,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:20,815 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-11 03:54:20,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-11 03:54:20,916 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:21,177 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 03:54:21,178 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:21,179 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,180 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:21,181 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:21,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:21,430 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:21,430 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:21,431 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:21,432 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 03:54:21,433 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:21,433 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:21,433 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:21,434 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:21,434 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,434 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:21,434 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,434 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,435 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,435 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:21,435 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,435 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,436 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:21,436 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,436 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:21,436 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,436 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,436 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,436 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:21,437 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:21,437 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:21,438 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087691438 2023-02-11 03:54:21,439 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,439 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:21,441 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:21,442 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:21,442 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:21,442 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087721442 2023-02-11 03:54:21,442 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087781442 2023-02-11 03:54:21,442 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:21,442 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:21,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:21,444 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:21,446 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:21,692 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,692 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:21,692 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:21,692 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:21,693 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:21,694 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:21,694 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:21,694 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:21,694 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:21,694 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:21,695 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 03:54:21,696 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:21,696 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:21,696 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:21,696 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:21,697 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,697 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:21,697 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,697 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,698 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,698 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:21,698 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,698 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,698 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:21,699 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,699 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:21,699 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,699 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,699 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,699 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:21,700 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:21,700 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:21,700 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087691700 2023-02-11 03:54:21,701 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:21,702 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,704 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:21,704 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:21,704 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:21,704 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087721704 2023-02-11 03:54:21,704 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087781704 2023-02-11 03:54:21,704 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:21,704 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:21,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN}] 2023-02-11 03:54:21,706 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:21,706 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:21,706 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:21,954 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,954 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:21,954 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:21,955 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:21,956 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:21,956 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:21,956 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:21,956 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:21,956 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:21,957 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:21,957 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 03:54:21,958 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:21,958 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:21,958 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:21,958 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:21,959 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,959 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:21,959 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,959 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:21,960 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,960 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:21,961 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,961 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:21,961 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:21,961 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,962 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:21,962 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,962 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,962 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,962 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:21,962 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:21,963 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:21,963 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087691963 2023-02-11 03:54:21,963 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:21,965 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:21,966 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:21,967 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:21,967 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:21,967 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087721967 2023-02-11 03:54:21,967 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087781967 2023-02-11 03:54:21,967 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:21,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN}] 2023-02-11 03:54:21,967 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:21,969 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:21,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:21,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087661969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087661969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087661969"}]},"ts":"1676087661969"} 2023-02-11 03:54:21,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:21,971 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:21,971 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:21,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:21,971 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:22,217 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,217 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:22,217 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:22,217 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:22,220 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:22,221 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:22,221 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:22,221 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:22,221 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:22,222 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:22,223 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:22,223 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:22,223 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:22,224 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:22,224 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:22,224 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:22,224 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,225 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:22,225 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,225 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,225 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,226 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:22,226 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,226 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,226 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:22,227 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,227 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,227 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,227 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,227 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,227 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,227 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,227 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,227 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:22,228 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:22,228 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:22,229 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:22,229 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087692229 2023-02-11 03:54:22,230 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,230 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:22,232 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:22,232 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:22,232 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:22,232 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087722232 2023-02-11 03:54:22,232 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087782232 2023-02-11 03:54:22,232 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:22,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,233 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087662232"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087662232"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087662232"}]},"ts":"1676087662232"} 2023-02-11 03:54:22,234 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:22,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:22,387 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:22,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a2d161464af91aca78ab4c3203d8177, disabling compactions & flushes 2023-02-11 03:54:22,388 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:22,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:22,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. after waiting 0 ms 2023-02-11 03:54:22,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:22,388 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 6a2d161464af91aca78ab4c3203d8177 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:54:22,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/2a4a46f9c5a740869db76bceadfd99d9 2023-02-11 03:54:22,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/63fd0baa4f3747f68b493b62e5450d71 2023-02-11 03:54:22,415 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/2a4a46f9c5a740869db76bceadfd99d9 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 2023-02-11 03:54:22,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:22,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/63fd0baa4f3747f68b493b62e5450d71 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 2023-02-11 03:54:22,424 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:54:22,425 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 37ms, sequenceid=82, compaction requested=true 2023-02-11 03:54:22,429 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:54:22,429 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:22,430 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:22,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a2d161464af91aca78ab4c3203d8177: Waiting for close lock at 1676087662388Running coprocessor pre-close hooks at 1676087662388Disabling compacts and flushes for region at 1676087662388Disabling writes for close at 1676087662388Obtaining lock to block concurrent updates at 1676087662388Preparing flush snapshotting stores in 6a2d161464af91aca78ab4c3203d8177 at 1676087662388Finished memstore snapshotting testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087662388Flushing stores of testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. at 1676087662389 (+1 ms)Flushing 6a2d161464af91aca78ab4c3203d8177/cf1: creating writer at 1676087662389Flushing 6a2d161464af91aca78ab4c3203d8177/cf1: appending metadata at 1676087662392 (+3 ms)Flushing 6a2d161464af91aca78ab4c3203d8177/cf1: closing flushed file at 1676087662392Flushing 6a2d161464af91aca78ab4c3203d8177/cf2: creating writer at 1676087662403 (+11 ms)Flushing 6a2d161464af91aca78ab4c3203d8177/cf2: appending metadata at 1676087662405 (+2 ms)Flushing 6a2d161464af91aca78ab4c3203d8177/cf2: closing flushed file at 1676087662405Flushing 6a2d161464af91aca78ab4c3203d8177/cf1: reopening flushed file at 1676087662416 (+11 ms)Flushing 6a2d161464af91aca78ab4c3203d8177/cf2: reopening flushed file at 1676087662421 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6a2d161464af91aca78ab4c3203d8177 in 37ms, sequenceid=82, compaction requested=true at 1676087662425 (+4 ms)Writing region close event to WAL at 1676087662426 (+1 ms)Running coprocessor post-close hooks at 1676087662429 (+3 ms)Closed at 1676087662430 (+1 ms) 2023-02-11 03:54:22,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:22,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED 2023-02-11 03:54:22,432 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087662432"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087662432"}]},"ts":"1676087662432"} 2023-02-11 03:54:22,433 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:22,434 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:22,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:22,482 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,482 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:22,483 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:22,483 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:22,484 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:22,485 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:22,485 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 03:54:22,485 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-11 03:54:22,485 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:22,486 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:22,486 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:22,486 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:22,487 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,487 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:22,487 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,488 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,488 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,488 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:22,488 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,489 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,489 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:22,490 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,490 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,490 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,490 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,490 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,490 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,490 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,491 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,491 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,491 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,491 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:54:22,492 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:22,492 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:22,493 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:22,493 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087692493 2023-02-11 03:54:22,494 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,494 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:22,497 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:22,497 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:22,497 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:22,497 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087722497 2023-02-11 03:54:22,498 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087782498 2023-02-11 03:54:22,498 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:22,498 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,498 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087662498"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087662498"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087662498"}]},"ts":"1676087662498"} 2023-02-11 03:54:22,500 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:22,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-11 03:54:22,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 in 266 msec 2023-02-11 03:54:22,503 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:22,503 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:22,503 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:22,748 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,748 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:22,748 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:22,748 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:22,749 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:22,749 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:22,749 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:22,749 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:22,750 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:22,750 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:22,751 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:22,751 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-11 03:54:22,751 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:22,752 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:22,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:22,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:22,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:22,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:22,754 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,754 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:22,754 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,754 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:22,754 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,755 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:22,755 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:22,756 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,756 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,756 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:22,756 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:22,756 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,756 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,756 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,757 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:22,757 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:22,757 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:54:22,757 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:22,758 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:22,758 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:22,758 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087692758 2023-02-11 03:54:22,758 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:22,759 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,762 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:22,762 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:22,762 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:22,762 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087722762 2023-02-11 03:54:22,762 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087782762 2023-02-11 03:54:22,762 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:22,762 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:22,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:22,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-11 03:54:22,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN in 797 msec 2023-02-11 03:54:22,771 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:22,771 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:22,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:22,771 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:23,012 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,013 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:23,013 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:23,013 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:23,014 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:23,014 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:23,014 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:23,014 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:23,014 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:23,015 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:23,016 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:23,016 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:23,016 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:23,016 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:23,017 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:23,017 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:23,017 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:23,018 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,018 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:23,018 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,019 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,019 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,019 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:23,020 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,020 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,020 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:23,021 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:23,022 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:54:23,022 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:23,023 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:23,023 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087693023 2023-02-11 03:54:23,025 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:23,025 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,027 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:23,027 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:23,027 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:23,028 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087723027 2023-02-11 03:54:23,028 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087783028 2023-02-11 03:54:23,028 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:23,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,028 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,032 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:23,041 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=6a2d161464af91aca78ab4c3203d8177, threads=14 2023-02-11 03:54:23,041 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,041 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,041 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,042 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,043 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,043 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,043 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,253 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,265 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,266 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,293 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,293 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,296 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,296 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,301 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,302 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,305 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,305 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,306 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,309 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,685 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,685 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 6a2d161464af91aca78ab4c3203d8177 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:23,717 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:23,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:23,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:23,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:23,778 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,778 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:23,778 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:23,779 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:23,780 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:23,780 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:23,780 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:23,780 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:23,780 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:23,781 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:23,782 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 03:54:23,782 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:23,782 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:23,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:23,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:23,783 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:23,784 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:23,784 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,784 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:23,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:23,785 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:23,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:23,786 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:23,787 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,787 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:23,787 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:23,787 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:23,788 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,788 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,788 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,788 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:23,788 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:23,788 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:54:23,789 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:23,789 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:23,790 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087693790 2023-02-11 03:54:23,790 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:23,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:23,794 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:23,794 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:23,794 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:23,794 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087723794 2023-02-11 03:54:23,794 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087783794 2023-02-11 03:54:23,794 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:23,803 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=6a2d161464af91aca78ab4c3203d8177, threads=14 2023-02-11 03:54:23,804 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,805 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,805 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,805 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,806 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,806 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,806 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,808 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,808 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,808 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,809 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,809 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,809 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,809 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,937 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,948 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,968 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,969 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,980 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,988 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:23,996 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,001 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,013 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,015 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,018 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,020 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,408 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,416 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 for region: 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:24,417 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 6a2d161464af91aca78ab4c3203d8177 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:24,453 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:24,458 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/2913b9be4358f3d94713905a911dbe4f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:24,460 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/92a7bf8419ade3abdc86f26cadcde207/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:54:24,460 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:24,460 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:24,460 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:24,545 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,545 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:24,545 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:24,545 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:24,546 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:24,546 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:24,546 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:24,546 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:24,546 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:24,547 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:24,548 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 03:54:24,548 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:24,548 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:24,548 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:24,548 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:24,549 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:24,549 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:24,549 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,549 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:24,550 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,550 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,550 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,550 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:24,550 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,551 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,551 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:24,551 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,551 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:24,551 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:24,553 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:24,553 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,553 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,553 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,553 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:24,553 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:24,554 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:54:24,554 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:24,555 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:24,555 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087694555 2023-02-11 03:54:24,555 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:24,556 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,558 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:24,558 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:24,558 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:24,558 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087724558 2023-02-11 03:54:24,558 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087784558 2023-02-11 03:54:24,558 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:24,561 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:24,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:24,563 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:24,563 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:24,809 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,809 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:24,809 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:24,809 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:24,810 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:24,810 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:24,810 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:24,811 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:24,811 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:24,811 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:24,812 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 03:54:24,812 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:24,812 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:24,812 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:24,813 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:24,813 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:24,813 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:24,813 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:24,814 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:24,815 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:24,815 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,816 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,816 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,816 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:24,816 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:24,816 DEBUG [Listener at localhost/45015] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,816 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:24,816 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:24,817 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:24,817 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087694817 2023-02-11 03:54:24,820 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:24,820 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:24,822 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:24,822 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:24,822 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:24,822 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087724822 2023-02-11 03:54:24,822 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087784822 2023-02-11 03:54:24,822 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:24,823 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:24,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:24,835 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:24,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:24,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:25,073 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:25,074 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:25,074 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:25,075 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:25,076 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 03:54:25,076 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:25,076 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:25,076 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:25,077 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:25,077 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:25,077 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:25,077 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,077 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,078 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,079 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:25,080 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,080 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,080 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,080 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,080 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:25,081 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:25,081 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:25,082 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087695082 2023-02-11 03:54:25,084 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:25,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,087 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:25,087 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:25,087 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:25,087 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087725087 2023-02-11 03:54:25,087 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087785087 2023-02-11 03:54:25,087 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:25,087 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:25,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:25,090 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:25,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:25,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:25,338 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:25,339 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:25,339 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:25,340 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:25,340 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:25,341 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 03:54:25,341 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:25,342 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:25,342 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:25,342 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:25,343 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:25,343 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:25,343 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,344 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:25,344 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,344 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,344 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,345 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:25,345 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,345 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,346 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:25,347 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,347 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:25,347 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:25,347 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:25,347 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,347 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,347 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,347 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,348 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,348 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:54:25,348 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:25,349 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:25,349 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087695349 2023-02-11 03:54:25,349 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:25,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,353 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:25,353 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:25,353 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:25,353 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087725353 2023-02-11 03:54:25,353 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087785353 2023-02-11 03:54:25,353 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:25,353 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:25,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:25,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:25,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:25,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:25,604 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,604 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:25,604 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:25,604 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:25,605 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:25,605 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:25,605 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:25,605 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:25,605 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:25,606 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:25,607 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 03:54:25,607 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:25,607 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:25,607 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:25,608 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:25,608 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:25,608 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:25,609 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,609 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:25,609 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,609 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,610 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,610 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:25,610 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,610 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,610 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:25,611 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,611 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,611 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,611 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,612 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:25,612 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:25,612 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:25,613 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087695613 2023-02-11 03:54:25,614 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:25,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,617 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:25,617 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:25,617 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:25,617 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087725617 2023-02-11 03:54:25,617 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087785617 2023-02-11 03:54:25,617 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:54:25,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:25,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN] 2023-02-11 03:54:25,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:25,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:25,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:25,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:25,867 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,867 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:25,867 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:25,868 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:25,869 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:25,869 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:25,869 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:25,869 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:25,869 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:25,870 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:25,870 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 03:54:25,870 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:54:25,870 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:54:25,870 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:25,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:25,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:25,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:25,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,872 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:25,873 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,873 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:25,873 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:25,874 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,874 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,874 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,874 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,875 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,875 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:25,875 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:25,875 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:25,876 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:25,876 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:25,876 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087695876 2023-02-11 03:54:25,876 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:25,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:25,879 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:25,881 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:25,881 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:25,881 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:25,881 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087725881 2023-02-11 03:54:25,881 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087785881 2023-02-11 03:54:25,881 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 03:54:25,881 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:25,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:25,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:25,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:25,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:26,131 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,132 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:26,132 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:26,132 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:26,134 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:26,134 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:26,134 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:26,134 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:26,134 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:26,135 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:26,136 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:54:26,136 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:26,136 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:26,136 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,137 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:26,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:26,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,139 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:26,139 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,139 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,139 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,139 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:26,139 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:26,140 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:26,140 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:26,141 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:26,141 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087696141 2023-02-11 03:54:26,142 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:26,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,144 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,145 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:26,146 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:26,146 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:26,146 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087726146 2023-02-11 03:54:26,146 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087786146 2023-02-11 03:54:26,146 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:26,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:26,149 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:26,149 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:26,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:26,149 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:26,396 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:26,397 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:26,397 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:26,398 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:26,399 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:26,400 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,400 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:26,400 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,400 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,401 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,401 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:26,401 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,401 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,401 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:26,402 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,402 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,402 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:26,402 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,402 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,402 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,402 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:26,402 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:26,403 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:26,403 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:26,403 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:26,403 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087696403 2023-02-11 03:54:26,404 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:26,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,406 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,408 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:26,409 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:26,409 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:26,409 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087726409 2023-02-11 03:54:26,409 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087786409 2023-02-11 03:54:26,409 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 03:54:26,409 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:26,411 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:26,411 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-11 03:54:26,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:26,411 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:26,659 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,659 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:26,659 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:26,659 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:26,660 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:26,660 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:26,660 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:26,660 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:26,660 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:26,661 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:26,661 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 03:54:26,661 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-11 03:54:26,662 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:26,662 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:26,662 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:26,662 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:26,663 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,663 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:26,663 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,663 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,663 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,664 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:26,664 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,664 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,664 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:26,665 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,665 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,665 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:26,665 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,665 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,665 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,665 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:54:26,665 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-11 03:54:26,666 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:26,666 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:26,666 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:26,667 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087696667 2023-02-11 03:54:26,669 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,669 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:26,671 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,673 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:26,673 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:26,673 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:26,673 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087726673 2023-02-11 03:54:26,674 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087786674 2023-02-11 03:54:26,674 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:54:26,674 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:54:26,676 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 exec-time=5.4960 sec 2023-02-11 03:54:26,677 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,677 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=true 2023-02-11 03:54:26,828 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:26,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:26,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087666829"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087666829"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087666829"}]},"ts":"1676087666829"} 2023-02-11 03:54:26,830 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:26,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:54:26,831 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:54:26,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:54:26,924 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:26,924 WARN [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-11 03:54:26,924 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:54:26,924 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:54:26,924 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:54:26,924 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:54:26,924 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:54:26,925 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:54:26,925 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:54:26,925 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:54:26,925 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:54:26,926 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-11 03:54:26,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:54:26,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:54:26,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:54:26,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:54:26,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:54:26,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:54:26,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:54:26,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,930 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:54:26,930 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:26,930 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:54:26,930 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,931 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:54:26,931 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,931 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-11 03:54:26,931 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:54:26,932 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:54:26,932 INFO [Listener at localhost/45015] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177 to restore RIT 2023-02-11 03:54:26,932 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087696932 2023-02-11 03:54:26,933 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:54:26,933 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:54:26,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:54:26,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=6a2d161464af91aca78ab4c3203d8177, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:54:26,935 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:54:26,936 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087726936 2023-02-11 03:54:26,936 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087786936 2023-02-11 03:54:26,936 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:54:26,936 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:26,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087666936"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087666936"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087666936"}]},"ts":"1676087666936"} 2023-02-11 03:54:26,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:27,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a2d161464af91aca78ab4c3203d8177, NAME => 'testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:27,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:27,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:27,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,096 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,097 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1 2023-02-11 03:54:27,097 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1 2023-02-11 03:54:27,097 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a2d161464af91aca78ab4c3203d8177 columnFamilyName cf1 2023-02-11 03:54:27,106 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 2023-02-11 03:54:27,109 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c 2023-02-11 03:54:27,113 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 2023-02-11 03:54:27,117 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 2023-02-11 03:54:27,120 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e 2023-02-11 03:54:27,129 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 2023-02-11 03:54:27,132 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 2023-02-11 03:54:27,132 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(310): Store=6a2d161464af91aca78ab4c3203d8177/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:27,132 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,133 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2 2023-02-11 03:54:27,133 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2 2023-02-11 03:54:27,134 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a2d161464af91aca78ab4c3203d8177 columnFamilyName cf2 2023-02-11 03:54:27,140 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c 2023-02-11 03:54:27,144 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 2023-02-11 03:54:27,147 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 2023-02-11 03:54:27,150 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 2023-02-11 03:54:27,153 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b 2023-02-11 03:54:27,156 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 2023-02-11 03:54:27,160 DEBUG [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa 2023-02-11 03:54:27,160 INFO [StoreOpener-6a2d161464af91aca78ab4c3203d8177-1] regionserver.HStore(310): Store=6a2d161464af91aca78ab4c3203d8177/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:27,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.splits 2023-02-11 03:54:27,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:27,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a2d161464af91aca78ab4c3203d8177; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66223348, jitterRate=-0.013195216655731201}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:27,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a2d161464af91aca78ab4c3203d8177: Running coprocessor pre-open hook at 1676087667095Writing region info on filesystem at 1676087667095Initializing all the Stores at 1676087667095Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087667095Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087667095Cleaning up temporary data from old regions at 1676087667162 (+67 ms)Cleaning up detritus from prior splits at 1676087667162Running coprocessor post-open hooks at 1676087667166 (+4 ms)Region opened successfully at 1676087667166 2023-02-11 03:54:27,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177., pid=48, masterSystemTime=1676087667090 2023-02-11 03:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:27,168 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:27,168 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:27,169 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 03:54:27,169 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 03:54:27,169 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): 6a2d161464af91aca78ab4c3203d8177/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:27,170 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): 6a2d161464af91aca78ab4c3203d8177/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:27,170 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a2d161464af91aca78ab4c3203d8177/cf2 in testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,170 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a2d161464af91aca78ab4c3203d8177/cf1 in testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,170 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp, totalSize=34.9 K 2023-02-11 03:54:27,170 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp, totalSize=34.9 K 2023-02-11 03:54:27,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,170 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 2587059ea4c648fca240f5cc07e382a8, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087659739 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting f94605cc975f45e1986f1d4d481a2bf3, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087659739 2023-02-11 03:54:27,171 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087667170"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087667170"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087667170"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087667170"}]},"ts":"1676087667170"} 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 21f16c1505e5466282edb372ef21bc77, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087659916 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 62228d4a95504ab4be8ae96e7377cbf3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087659916 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting a175abc0fc44470ba56639f6fc6ae59b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087660126 2023-02-11 03:54:27,171 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting cb07deae2fcd4fe5af64fe6b950af447, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087660126 2023-02-11 03:54:27,172 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting c082ae972b4e460cb0ca58be7cd7abc4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087660333 2023-02-11 03:54:27,172 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 9b4d2b46c1d84ad089e37360e4e0de1e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087660333 2023-02-11 03:54:27,172 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 0c0fd9cd6e7d4fbb9880a4cbc7b1e96c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087660516 2023-02-11 03:54:27,172 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 319b67b292c344929e2b13bef796764c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087660516 2023-02-11 03:54:27,173 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting e04e081a3651417abde5953b14c8c0fa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087660711 2023-02-11 03:54:27,173 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 88f9802016fb451fb1e48baf455660d9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087660711 2023-02-11 03:54:27,173 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 63fd0baa4f3747f68b493b62e5450d71, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087660918 2023-02-11 03:54:27,173 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 2a4a46f9c5a740869db76bceadfd99d9, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087660918 2023-02-11 03:54:27,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-11 03:54:27,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 in 234 msec 2023-02-11 03:54:27,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN in 1.5560 sec 2023-02-11 03:54:27,187 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-11 03:54:27,190 DEBUG [Listener at localhost/45015] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-11 03:54:27,192 INFO [Listener at localhost/45015] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 03:54:27,197 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6a2d161464af91aca78ab4c3203d8177#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:27,197 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 6a2d161464af91aca78ab4c3203d8177#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:27,198 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-11 03:54:27,198 INFO [Listener at localhost/45015] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-11 03:54:27,198 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-11 03:54:27,226 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf1/0b4e82a593234b319c36098ec64abaa7 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/0b4e82a593234b319c36098ec64abaa7 2023-02-11 03:54:27,227 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/.tmp/cf2/41525012ff8948078e7573119bb64dc8 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/41525012ff8948078e7573119bb64dc8 2023-02-11 03:54:27,238 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6a2d161464af91aca78ab4c3203d8177/cf1 of 6a2d161464af91aca78ab4c3203d8177 into 0b4e82a593234b319c36098ec64abaa7(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:27,238 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:27,238 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177., storeName=6a2d161464af91aca78ab4c3203d8177/cf1, priority=9, startTime=1676087667167; duration=0sec 2023-02-11 03:54:27,239 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:27,239 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6a2d161464af91aca78ab4c3203d8177/cf2 of 6a2d161464af91aca78ab4c3203d8177 into 41525012ff8948078e7573119bb64dc8(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:27,239 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a2d161464af91aca78ab4c3203d8177: 2023-02-11 03:54:27,239 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177., storeName=6a2d161464af91aca78ab4c3203d8177/cf2, priority=9, startTime=1676087667167; duration=0sec 2023-02-11 03:54:27,242 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:27,247 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:27,249 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-11 03:54:27,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-11 03:54:27,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 03:54:27,254 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087667254"}]},"ts":"1676087667254"} 2023-02-11 03:54:27,255 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 03:54:27,257 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-11 03:54:27,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN}] 2023-02-11 03:54:27,259 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN 2023-02-11 03:54:27,259 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:27,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087667259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087667259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087667259"}]},"ts":"1676087667259"} 2023-02-11 03:54:27,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:27,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 03:54:27,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a2d161464af91aca78ab4c3203d8177, disabling compactions & flushes 2023-02-11 03:54:27,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. after waiting 0 ms 2023-02-11 03:54:27,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,414 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9] to archive 2023-02-11 03:54:27,415 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:27,416 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/f94605cc975f45e1986f1d4d481a2bf3 2023-02-11 03:54:27,418 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/62228d4a95504ab4be8ae96e7377cbf3 2023-02-11 03:54:27,419 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/cb07deae2fcd4fe5af64fe6b950af447 2023-02-11 03:54:27,420 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/9b4d2b46c1d84ad089e37360e4e0de1e 2023-02-11 03:54:27,421 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/319b67b292c344929e2b13bef796764c 2023-02-11 03:54:27,423 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/88f9802016fb451fb1e48baf455660d9 2023-02-11 03:54:27,424 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/2a4a46f9c5a740869db76bceadfd99d9 2023-02-11 03:54:27,428 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71] to archive 2023-02-11 03:54:27,429 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:27,430 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/2587059ea4c648fca240f5cc07e382a8 2023-02-11 03:54:27,431 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/21f16c1505e5466282edb372ef21bc77 2023-02-11 03:54:27,432 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/a175abc0fc44470ba56639f6fc6ae59b 2023-02-11 03:54:27,433 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/c082ae972b4e460cb0ca58be7cd7abc4 2023-02-11 03:54:27,434 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/0c0fd9cd6e7d4fbb9880a4cbc7b1e96c 2023-02-11 03:54:27,435 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/e04e081a3651417abde5953b14c8c0fa 2023-02-11 03:54:27,436 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/63fd0baa4f3747f68b493b62e5450d71 2023-02-11 03:54:27,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:54:27,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:27,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177. 2023-02-11 03:54:27,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a2d161464af91aca78ab4c3203d8177: Waiting for close lock at 1676087667413Running coprocessor pre-close hooks at 1676087667413Disabling compacts and flushes for region at 1676087667413Disabling writes for close at 1676087667413Writing region close event to WAL at 1676087667437 (+24 ms)Running coprocessor post-close hooks at 1676087667440 (+3 ms)Closed at 1676087667440 2023-02-11 03:54:27,441 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,442 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=6a2d161464af91aca78ab4c3203d8177, regionState=CLOSED 2023-02-11 03:54:27,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087667442"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087667442"}]},"ts":"1676087667442"} 2023-02-11 03:54:27,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-11 03:54:27,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 6a2d161464af91aca78ab4c3203d8177, server=jenkins-hbase4.apache.org,36661,1676087644839 in 182 msec 2023-02-11 03:54:27,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-11 03:54:27,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, UNASSIGN in 187 msec 2023-02-11 03:54:27,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087667447"}]},"ts":"1676087667447"} 2023-02-11 03:54:27,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 03:54:27,450 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-11 03:54:27,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 201 msec 2023-02-11 03:54:27,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 03:54:27,556 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-11 03:54:27,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-11 03:54:27,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,559 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,560 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 03:54:27,563 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,565 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits] 2023-02-11 03:54:27,567 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/0b4e82a593234b319c36098ec64abaa7 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf1/0b4e82a593234b319c36098ec64abaa7 2023-02-11 03:54:27,569 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/41525012ff8948078e7573119bb64dc8 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/cf2/41525012ff8948078e7573119bb64dc8 2023-02-11 03:54:27,572 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177/recovered.edits/90.seqid 2023-02-11 03:54:27,572 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackAndDoubleExecution/6a2d161464af91aca78ab4c3203d8177 2023-02-11 03:54:27,572 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-11 03:54:27,575 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,577 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-11 03:54:27,578 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-11 03:54:27,579 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,579 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-11 03:54:27,579 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087667579"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:27,581 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 03:54:27,581 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6a2d161464af91aca78ab4c3203d8177, NAME => 'testRollbackAndDoubleExecution,,1676087659231.6a2d161464af91aca78ab4c3203d8177.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 03:54:27,581 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-11 03:54:27,581 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087667581"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:27,583 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-11 03:54:27,584 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:54:27,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 27 msec 2023-02-11 03:54:27,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-11 03:54:27,661 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-11 03:54:27,678 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398 (was 394) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45612 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50550 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59742 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59728 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50470 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59708 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:36661-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50496 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45644 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45626 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59726 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45636 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50488 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45598 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45582 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50482 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59722 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=861 (was 843) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 231), ProcessCount=168 (was 168), AvailableMemoryMB=3261 (was 3297) 2023-02-11 03:54:27,694 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=398, OpenFileDescriptor=861, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=168, AvailableMemoryMB=3261 2023-02-11 03:54:27,695 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:27,698 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:27,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:27,700 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:27,701 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:27,702 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:27,703 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:27,704 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 empty. 2023-02-11 03:54:27,704 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:27,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 03:54:27,714 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:27,715 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => d5f2b752b9b4350ebebfba4a4f344067, NAME => 'testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing d5f2b752b9b4350ebebfba4a4f344067, disabling compactions & flushes 2023-02-11 03:54:27,730 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. after waiting 0 ms 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:27,730 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:27,730 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for d5f2b752b9b4350ebebfba4a4f344067: Waiting for close lock at 1676087667730Disabling compacts and flushes for region at 1676087667730Disabling writes for close at 1676087667730Writing region close event to WAL at 1676087667730Closed at 1676087667730 2023-02-11 03:54:27,732 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:27,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087667733"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087667733"}]},"ts":"1676087667733"} 2023-02-11 03:54:27,735 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:27,735 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:27,736 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087667736"}]},"ts":"1676087667736"} 2023-02-11 03:54:27,737 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-11 03:54:27,740 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:27,741 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:27,741 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:27,741 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:27,741 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:27,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, ASSIGN}] 2023-02-11 03:54:27,743 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, ASSIGN 2023-02-11 03:54:27,743 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:27,893 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:27,894 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=d5f2b752b9b4350ebebfba4a4f344067, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:27,895 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087667894"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087667894"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087667894"}]},"ts":"1676087667894"} 2023-02-11 03:54:27,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure d5f2b752b9b4350ebebfba4a4f344067, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:28,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d5f2b752b9b4350ebebfba4a4f344067, NAME => 'testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:28,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:28,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,052 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,053 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,054 DEBUG [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf1 2023-02-11 03:54:28,054 DEBUG [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf1 2023-02-11 03:54:28,054 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5f2b752b9b4350ebebfba4a4f344067 columnFamilyName cf1 2023-02-11 03:54:28,055 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] regionserver.HStore(310): Store=d5f2b752b9b4350ebebfba4a4f344067/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,055 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,056 DEBUG [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf2 2023-02-11 03:54:28,056 DEBUG [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf2 2023-02-11 03:54:28,057 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d5f2b752b9b4350ebebfba4a4f344067 columnFamilyName cf2 2023-02-11 03:54:28,057 INFO [StoreOpener-d5f2b752b9b4350ebebfba4a4f344067-1] regionserver.HStore(310): Store=d5f2b752b9b4350ebebfba4a4f344067/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:28,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:28,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d5f2b752b9b4350ebebfba4a4f344067; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75247891, jitterRate=0.12128095328807831}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:28,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d5f2b752b9b4350ebebfba4a4f344067: Running coprocessor pre-open hook at 1676087668052Writing region info on filesystem at 1676087668052Initializing all the Stores at 1676087668052Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668052Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668053 (+1 ms)Cleaning up temporary data from old regions at 1676087668059 (+6 ms)Cleaning up detritus from prior splits at 1676087668059Running coprocessor post-open hooks at 1676087668064 (+5 ms)Region opened successfully at 1676087668064 2023-02-11 03:54:28,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067., pid=55, masterSystemTime=1676087668048 2023-02-11 03:54:28,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,066 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=d5f2b752b9b4350ebebfba4a4f344067, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,066 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087668066"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087668066"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087668066"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087668066"}]},"ts":"1676087668066"} 2023-02-11 03:54:28,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-11 03:54:28,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure d5f2b752b9b4350ebebfba4a4f344067, server=jenkins-hbase4.apache.org,36661,1676087644839 in 172 msec 2023-02-11 03:54:28,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-11 03:54:28,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, ASSIGN in 328 msec 2023-02-11 03:54:28,071 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:28,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087668071"}]},"ts":"1676087668071"} 2023-02-11 03:54:28,073 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-11 03:54:28,075 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:28,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 376 msec 2023-02-11 03:54:28,201 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,202 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,203 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,203 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,203 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, UNASSIGN}] 2023-02-11 03:54:28,205 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, UNASSIGN 2023-02-11 03:54:28,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=d5f2b752b9b4350ebebfba4a4f344067, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087668206"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087668206"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087668206"}]},"ts":"1676087668206"} 2023-02-11 03:54:28,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure d5f2b752b9b4350ebebfba4a4f344067, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:28,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d5f2b752b9b4350ebebfba4a4f344067, disabling compactions & flushes 2023-02-11 03:54:28,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. after waiting 0 ms 2023-02-11 03:54:28,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 03:54:28,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:28,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067. 2023-02-11 03:54:28,365 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d5f2b752b9b4350ebebfba4a4f344067: Waiting for close lock at 1676087668359Running coprocessor pre-close hooks at 1676087668359Disabling compacts and flushes for region at 1676087668359Disabling writes for close at 1676087668360 (+1 ms)Writing region close event to WAL at 1676087668361 (+1 ms)Running coprocessor post-close hooks at 1676087668364 (+3 ms)Closed at 1676087668364 2023-02-11 03:54:28,366 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:28,366 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=d5f2b752b9b4350ebebfba4a4f344067, regionState=CLOSED 2023-02-11 03:54:28,366 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087668366"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087668366"}]},"ts":"1676087668366"} 2023-02-11 03:54:28,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-11 03:54:28,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure d5f2b752b9b4350ebebfba4a4f344067, server=jenkins-hbase4.apache.org,36661,1676087644839 in 161 msec 2023-02-11 03:54:28,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-11 03:54:28,371 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d5f2b752b9b4350ebebfba4a4f344067, UNASSIGN in 165 msec 2023-02-11 03:54:28,395 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 03:54:28,397 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-11 03:54:28,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087668398"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676087668398"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676087668398"}]},"ts":"1676087668398"} 2023-02-11 03:54:28,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087668398"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087668398"}]},"ts":"1676087668398"} 2023-02-11 03:54:28,399 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087668398"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087668398"}]},"ts":"1676087668398"} 2023-02-11 03:54:28,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, ASSIGN}] 2023-02-11 03:54:28,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, ASSIGN 2023-02-11 03:54:28,403 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:28,403 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, ASSIGN 2023-02-11 03:54:28,404 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:28,553 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:54:28,555 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=51be33cfde98666a03236a2e1cc0ce5e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,555 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087668555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087668555"}]},"ts":"1676087668555"} 2023-02-11 03:54:28,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 51be33cfde98666a03236a2e1cc0ce5e, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:28,559 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=d012fc16eb3b4eb095cb578eaf069954, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,559 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668558"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087668558"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087668558"}]},"ts":"1676087668558"} 2023-02-11 03:54:28,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure d012fc16eb3b4eb095cb578eaf069954, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:28,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:28,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d012fc16eb3b4eb095cb578eaf069954, NAME => 'testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 03:54:28,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:28,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:28,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,712 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,714 DEBUG [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf1 2023-02-11 03:54:28,714 DEBUG [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf1 2023-02-11 03:54:28,714 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d012fc16eb3b4eb095cb578eaf069954 columnFamilyName cf1 2023-02-11 03:54:28,715 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] regionserver.HStore(310): Store=d012fc16eb3b4eb095cb578eaf069954/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,715 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,716 DEBUG [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf2 2023-02-11 03:54:28,716 DEBUG [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf2 2023-02-11 03:54:28,716 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d012fc16eb3b4eb095cb578eaf069954 columnFamilyName cf2 2023-02-11 03:54:28,717 INFO [StoreOpener-d012fc16eb3b4eb095cb578eaf069954-1] regionserver.HStore(310): Store=d012fc16eb3b4eb095cb578eaf069954/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:28,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:28,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d012fc16eb3b4eb095cb578eaf069954; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63608415, jitterRate=-0.052160754799842834}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:28,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d012fc16eb3b4eb095cb578eaf069954: Running coprocessor pre-open hook at 1676087668711Writing region info on filesystem at 1676087668711Initializing all the Stores at 1676087668712 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668712Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668712Cleaning up temporary data from old regions at 1676087668719 (+7 ms)Cleaning up detritus from prior splits at 1676087668719Running coprocessor post-open hooks at 1676087668722 (+3 ms)Region opened successfully at 1676087668722 2023-02-11 03:54:28,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954., pid=62, masterSystemTime=1676087668708 2023-02-11 03:54:28,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:28,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:28,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:28,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 51be33cfde98666a03236a2e1cc0ce5e, NAME => 'testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 03:54:28,724 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=d012fc16eb3b4eb095cb578eaf069954, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:28,724 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668724"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087668724"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087668724"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087668724"}]},"ts":"1676087668724"} 2023-02-11 03:54:28,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:28,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,726 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,727 DEBUG [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf1 2023-02-11 03:54:28,727 DEBUG [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf1 2023-02-11 03:54:28,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-11 03:54:28,727 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure d012fc16eb3b4eb095cb578eaf069954, server=jenkins-hbase4.apache.org,36661,1676087644839 in 166 msec 2023-02-11 03:54:28,727 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51be33cfde98666a03236a2e1cc0ce5e columnFamilyName cf1 2023-02-11 03:54:28,728 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] regionserver.HStore(310): Store=51be33cfde98666a03236a2e1cc0ce5e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,728 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, ASSIGN in 325 msec 2023-02-11 03:54:28,729 DEBUG [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf2 2023-02-11 03:54:28,729 DEBUG [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf2 2023-02-11 03:54:28,729 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 51be33cfde98666a03236a2e1cc0ce5e columnFamilyName cf2 2023-02-11 03:54:28,730 INFO [StoreOpener-51be33cfde98666a03236a2e1cc0ce5e-1] regionserver.HStore(310): Store=51be33cfde98666a03236a2e1cc0ce5e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:28,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:28,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:28,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 51be33cfde98666a03236a2e1cc0ce5e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61789252, jitterRate=-0.07926839590072632}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:28,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 51be33cfde98666a03236a2e1cc0ce5e: Running coprocessor pre-open hook at 1676087668725Writing region info on filesystem at 1676087668725Initializing all the Stores at 1676087668725Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668725Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087668726 (+1 ms)Cleaning up temporary data from old regions at 1676087668732 (+6 ms)Cleaning up detritus from prior splits at 1676087668732Running coprocessor post-open hooks at 1676087668735 (+3 ms)Region opened successfully at 1676087668735 2023-02-11 03:54:28,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e., pid=61, masterSystemTime=1676087668708 2023-02-11 03:54:28,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:28,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:28,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=51be33cfde98666a03236a2e1cc0ce5e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,737 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668737"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087668737"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087668737"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087668737"}]},"ts":"1676087668737"} 2023-02-11 03:54:28,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-11 03:54:28,740 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 51be33cfde98666a03236a2e1cc0ce5e, server=jenkins-hbase4.apache.org,36661,1676087644839 in 182 msec 2023-02-11 03:54:28,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-11 03:54:28,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, ASSIGN in 338 msec 2023-02-11 03:54:28,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 in 541 msec 2023-02-11 03:54:28,956 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:54:28,959 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:28,960 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-11 03:54:28,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-11 03:54:28,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:28,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 03:54:28,964 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087668964"}]},"ts":"1676087668964"} 2023-02-11 03:54:28,965 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-11 03:54:28,966 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-11 03:54:28,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, UNASSIGN}] 2023-02-11 03:54:28,967 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, UNASSIGN 2023-02-11 03:54:28,968 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=d012fc16eb3b4eb095cb578eaf069954, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,968 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668968"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087668968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087668968"}]},"ts":"1676087668968"} 2023-02-11 03:54:28,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure d012fc16eb3b4eb095cb578eaf069954, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:28,971 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, UNASSIGN 2023-02-11 03:54:28,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=51be33cfde98666a03236a2e1cc0ce5e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:28,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087668972"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087668972"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087668972"}]},"ts":"1676087668972"} 2023-02-11 03:54:28,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 51be33cfde98666a03236a2e1cc0ce5e, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:29,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 03:54:29,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:29,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 51be33cfde98666a03236a2e1cc0ce5e, disabling compactions & flushes 2023-02-11 03:54:29,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:29,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:29,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. after waiting 0 ms 2023-02-11 03:54:29,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:29,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 03:54:29,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:29,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e. 2023-02-11 03:54:29,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 51be33cfde98666a03236a2e1cc0ce5e: Waiting for close lock at 1676087669121Running coprocessor pre-close hooks at 1676087669121Disabling compacts and flushes for region at 1676087669121Disabling writes for close at 1676087669122 (+1 ms)Writing region close event to WAL at 1676087669122Running coprocessor post-close hooks at 1676087669127 (+5 ms)Closed at 1676087669127 2023-02-11 03:54:29,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:29,129 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:29,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d012fc16eb3b4eb095cb578eaf069954, disabling compactions & flushes 2023-02-11 03:54:29,130 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:29,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:29,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. after waiting 0 ms 2023-02-11 03:54:29,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:29,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=51be33cfde98666a03236a2e1cc0ce5e, regionState=CLOSED 2023-02-11 03:54:29,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087669130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087669130"}]},"ts":"1676087669130"} 2023-02-11 03:54:29,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-11 03:54:29,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 51be33cfde98666a03236a2e1cc0ce5e, server=jenkins-hbase4.apache.org,36661,1676087644839 in 159 msec 2023-02-11 03:54:29,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=51be33cfde98666a03236a2e1cc0ce5e, UNASSIGN in 167 msec 2023-02-11 03:54:29,140 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 03:54:29,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:29,141 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954. 2023-02-11 03:54:29,141 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d012fc16eb3b4eb095cb578eaf069954: Waiting for close lock at 1676087669129Running coprocessor pre-close hooks at 1676087669129Disabling compacts and flushes for region at 1676087669129Disabling writes for close at 1676087669130 (+1 ms)Writing region close event to WAL at 1676087669130Running coprocessor post-close hooks at 1676087669141 (+11 ms)Closed at 1676087669141 2023-02-11 03:54:29,143 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:29,143 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=d012fc16eb3b4eb095cb578eaf069954, regionState=CLOSED 2023-02-11 03:54:29,143 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087669143"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087669143"}]},"ts":"1676087669143"} 2023-02-11 03:54:29,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-11 03:54:29,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure d012fc16eb3b4eb095cb578eaf069954, server=jenkins-hbase4.apache.org,36661,1676087644839 in 175 msec 2023-02-11 03:54:29,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-11 03:54:29,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=d012fc16eb3b4eb095cb578eaf069954, UNASSIGN in 179 msec 2023-02-11 03:54:29,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087669149"}]},"ts":"1676087669149"} 2023-02-11 03:54:29,150 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-11 03:54:29,152 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-11 03:54:29,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 192 msec 2023-02-11 03:54:29,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-11 03:54:29,266 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-11 03:54:29,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 03:54:29,274 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:29,274 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:29,274 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:29,276 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/recovered.edits] 2023-02-11 03:54:29,276 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/recovered.edits] 2023-02-11 03:54:29,276 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/recovered.edits] 2023-02-11 03:54:29,283 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/recovered.edits/4.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067/recovered.edits/4.seqid 2023-02-11 03:54:29,283 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/recovered.edits/7.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e/recovered.edits/7.seqid 2023-02-11 03:54:29,283 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/recovered.edits/7.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954/recovered.edits/7.seqid 2023-02-11 03:54:29,284 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d5f2b752b9b4350ebebfba4a4f344067 2023-02-11 03:54:29,284 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/51be33cfde98666a03236a2e1cc0ce5e 2023-02-11 03:54:29,284 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionNoStoreFile/d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:54:29,284 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-11 03:54:29,287 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,288 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-11 03:54:29,290 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-11 03:54:29,291 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,291 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-11 03:54:29,291 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087669291"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:29,291 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087669291"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:29,291 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087669291"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:29,292 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:54:29,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d5f2b752b9b4350ebebfba4a4f344067, NAME => 'testSplitTableRegionNoStoreFile,,1676087667699.d5f2b752b9b4350ebebfba4a4f344067.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 51be33cfde98666a03236a2e1cc0ce5e, NAME => 'testSplitTableRegionNoStoreFile,,1676087668201.51be33cfde98666a03236a2e1cc0ce5e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => d012fc16eb3b4eb095cb578eaf069954, NAME => 'testSplitTableRegionNoStoreFile,41,1676087668201.d012fc16eb3b4eb095cb578eaf069954.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 03:54:29,292 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-11 03:54:29,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087669292"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:29,294 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-11 03:54:29,297 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:54:29,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 31 msec 2023-02-11 03:54:29,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-11 03:54:29,371 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-11 03:54:29,390 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401 (was 398) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1783315622_15 at /127.0.0.1:45612 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1783315622_15 at /127.0.0.1:50470 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=859 (was 861), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 202), ProcessCount=168 (was 168), AvailableMemoryMB=3259 (was 3261) 2023-02-11 03:54:29,408 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=401, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=202, ProcessCount=168, AvailableMemoryMB=3258 2023-02-11 03:54:29,408 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:29,411 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:29,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:29,413 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,415 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:29,416 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:29,417 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,418 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 empty. 2023-02-11 03:54:29,418 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,418 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 03:54:29,433 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:29,435 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 99a53710f43740e3cff337d6c3cb3857, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 99a53710f43740e3cff337d6c3cb3857, disabling compactions & flushes 2023-02-11 03:54:29,444 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. after waiting 0 ms 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,444 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,444 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 99a53710f43740e3cff337d6c3cb3857: Waiting for close lock at 1676087669444Disabling compacts and flushes for region at 1676087669444Disabling writes for close at 1676087669444Writing region close event to WAL at 1676087669444Closed at 1676087669444 2023-02-11 03:54:29,446 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:29,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087669447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087669447"}]},"ts":"1676087669447"} 2023-02-11 03:54:29,449 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:29,449 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:29,450 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087669450"}]},"ts":"1676087669450"} 2023-02-11 03:54:29,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-11 03:54:29,453 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:29,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:29,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:29,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:29,454 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:29,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, ASSIGN}] 2023-02-11 03:54:29,456 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, ASSIGN 2023-02-11 03:54:29,457 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:29,607 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:29,608 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=99a53710f43740e3cff337d6c3cb3857, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,608 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087669608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087669608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087669608"}]},"ts":"1676087669608"} 2023-02-11 03:54:29,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 99a53710f43740e3cff337d6c3cb3857, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:29,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 99a53710f43740e3cff337d6c3cb3857, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:29,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:29,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:29,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,766 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,767 DEBUG [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1 2023-02-11 03:54:29,767 DEBUG [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1 2023-02-11 03:54:29,768 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99a53710f43740e3cff337d6c3cb3857 columnFamilyName cf1 2023-02-11 03:54:29,768 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] regionserver.HStore(310): Store=99a53710f43740e3cff337d6c3cb3857/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:29,769 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,770 DEBUG [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2 2023-02-11 03:54:29,770 DEBUG [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2 2023-02-11 03:54:29,771 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99a53710f43740e3cff337d6c3cb3857 columnFamilyName cf2 2023-02-11 03:54:29,771 INFO [StoreOpener-99a53710f43740e3cff337d6c3cb3857-1] regionserver.HStore(310): Store=99a53710f43740e3cff337d6c3cb3857/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:29,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:29,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:29,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:29,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 99a53710f43740e3cff337d6c3cb3857; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67370855, jitterRate=0.0039039701223373413}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:29,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 99a53710f43740e3cff337d6c3cb3857: Running coprocessor pre-open hook at 1676087669765Writing region info on filesystem at 1676087669765Initializing all the Stores at 1676087669766 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087669766Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087669766Cleaning up temporary data from old regions at 1676087669773 (+7 ms)Cleaning up detritus from prior splits at 1676087669774 (+1 ms)Running coprocessor post-open hooks at 1676087669780 (+6 ms)Region opened successfully at 1676087669780 2023-02-11 03:54:29,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857., pid=71, masterSystemTime=1676087669761 2023-02-11 03:54:29,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,782 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=99a53710f43740e3cff337d6c3cb3857, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,782 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087669782"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087669782"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087669782"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087669782"}]},"ts":"1676087669782"} 2023-02-11 03:54:29,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-11 03:54:29,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 99a53710f43740e3cff337d6c3cb3857, server=jenkins-hbase4.apache.org,36661,1676087644839 in 174 msec 2023-02-11 03:54:29,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-11 03:54:29,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, ASSIGN in 331 msec 2023-02-11 03:54:29,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:29,787 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087669787"}]},"ts":"1676087669787"} 2023-02-11 03:54:29,788 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-11 03:54:29,791 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:29,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 379 msec 2023-02-11 03:54:29,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:29,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,922 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:29,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:29,923 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:29,923 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,923 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,924 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,924 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:29,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:29,926 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:29,926 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,927 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:29,927 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:54:29,928 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,929 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,929 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:29,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:29,937 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/7ff2365cf96447a4bb2cb2676f480b94 2023-02-11 03:54:29,952 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/60e53c5965ac4c0abb7b30017e402044 2023-02-11 03:54:29,956 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/7ff2365cf96447a4bb2cb2676f480b94 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94 2023-02-11 03:54:29,960 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:29,961 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/60e53c5965ac4c0abb7b30017e402044 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044 2023-02-11 03:54:29,964 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:54:29,965 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 38ms, sequenceid=6, compaction requested=false 2023-02-11 03:54:29,965 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:29,965 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:29,966 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:29,966 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:29,966 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:29,966 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,966 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:29,966 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:29,968 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,968 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,968 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,968 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:29,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:29,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@90c5e44[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:29,970 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:29,970 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:29,971 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,972 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:29,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:29,972 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:29,973 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,974 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:29,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,976 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:29,978 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:29,978 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:29,978 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:29,978 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:29,978 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,978 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,980 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,981 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,981 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,982 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,983 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,983 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,988 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:29,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:29,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,988 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:29,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:29,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,989 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,990 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:29,990 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:29,990 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:29,990 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,089 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,106 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:30,106 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:30,107 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,107 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,109 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,109 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,109 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,110 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,110 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,110 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,111 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,111 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:30,111 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:30,112 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,113 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,113 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,114 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,114 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,114 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,120 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 2023-02-11 03:54:30,151 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/61fa2d760ffe4a758120753eba86ca03 2023-02-11 03:54:30,156 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 2023-02-11 03:54:30,161 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:30,162 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/61fa2d760ffe4a758120753eba86ca03 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03 2023-02-11 03:54:30,165 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:54:30,166 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 55ms, sequenceid=19, compaction requested=false 2023-02-11 03:54:30,166 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:30,166 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,166 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,167 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,168 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,168 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,168 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,168 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:30,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c2ce206[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,170 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:30,170 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:30,172 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,175 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,175 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:30,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,178 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:30,178 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,178 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,178 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,178 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,179 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,179 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,179 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,180 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,180 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,181 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,183 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,183 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,183 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,186 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,186 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:30,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,287 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,303 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,303 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,303 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:30,303 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:30,303 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,303 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,304 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,304 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,304 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,304 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,305 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,305 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,306 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,306 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,306 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,307 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,307 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:30,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,307 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:30,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,308 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,307 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,308 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,311 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,311 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,311 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,311 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,312 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,312 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,323 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/4664e026d35f40f9952b9a69386f05e5 2023-02-11 03:54:30,335 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 2023-02-11 03:54:30,339 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/4664e026d35f40f9952b9a69386f05e5 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5 2023-02-11 03:54:30,343 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:30,344 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 2023-02-11 03:54:30,348 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:54:30,349 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 42ms, sequenceid=32, compaction requested=true 2023-02-11 03:54:30,349 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:30,349 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,349 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,350 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,350 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,350 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,350 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,354 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,354 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,354 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,354 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:30,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ce703f9[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,356 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:30,356 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,357 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,357 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,358 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,358 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:30,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,358 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:30,359 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,360 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,360 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,360 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,360 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,360 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,360 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,361 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:30,363 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,363 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:30,363 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,363 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,363 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,363 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,367 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,367 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,369 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,369 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,369 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,373 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,372 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,373 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,374 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,374 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,374 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,473 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,489 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,489 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,489 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:30,489 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:30,489 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,489 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,491 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,491 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,491 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,491 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,492 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,493 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,493 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,493 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,493 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,493 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,493 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,493 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:30,494 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:30,495 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,496 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,497 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,497 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,503 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/505212d797774861be3ca98bf11d7ce1 2023-02-11 03:54:30,515 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/e13122d44e9947c7a14e8cb2b59e1704 2023-02-11 03:54:30,543 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/505212d797774861be3ca98bf11d7ce1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1 2023-02-11 03:54:30,547 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:30,548 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/e13122d44e9947c7a14e8cb2b59e1704 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704 2023-02-11 03:54:30,551 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:54:30,552 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 59ms, sequenceid=45, compaction requested=true 2023-02-11 03:54:30,552 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:30,552 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,552 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,552 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,552 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,552 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,552 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,554 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,554 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:30,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f80a1d8[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,556 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:30,556 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,557 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,558 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,557 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:30,558 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:30,559 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,559 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,559 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,560 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,560 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:30,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,563 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:30,563 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,563 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,563 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,563 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,568 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,568 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,569 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,570 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,571 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,571 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,573 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,573 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,574 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,674 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,690 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:30,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:30,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,692 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,692 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,694 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,694 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,694 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,694 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,694 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,695 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,695 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,695 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:30,695 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,697 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,706 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/d3aea02c0afc4b20b821c1d6e9917f0d 2023-02-11 03:54:30,718 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/d51a0d4ff95b4a0292db24d38e56244a 2023-02-11 03:54:30,722 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/d3aea02c0afc4b20b821c1d6e9917f0d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d 2023-02-11 03:54:30,726 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:30,727 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/d51a0d4ff95b4a0292db24d38e56244a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a 2023-02-11 03:54:30,731 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:54:30,732 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 37ms, sequenceid=58, compaction requested=true 2023-02-11 03:54:30,733 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:30,733 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,733 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,735 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,735 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,735 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,735 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:30,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54152612[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,737 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:30,738 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,743 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:30,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,743 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:30,745 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,745 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,745 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,746 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,746 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,746 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,749 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:30,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,754 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:30,754 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,754 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,754 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,754 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,757 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,758 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,759 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,764 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:30,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,764 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,765 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,766 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,766 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,865 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,881 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,881 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,881 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:30,881 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:30,881 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,881 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,885 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,886 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,886 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,886 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:30,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,887 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,887 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,887 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:30,888 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:54:30,890 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,890 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,890 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,891 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,891 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,896 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/bff28646e8ec418dad11d4bad0f7eadc 2023-02-11 03:54:30,907 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/dd0d908cd73d4c36971d8e66a7e0965c 2023-02-11 03:54:30,911 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/bff28646e8ec418dad11d4bad0f7eadc as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc 2023-02-11 03:54:30,914 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:30,915 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/dd0d908cd73d4c36971d8e66a7e0965c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c 2023-02-11 03:54:30,918 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:54:30,919 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 31ms, sequenceid=71, compaction requested=true 2023-02-11 03:54:30,919 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:30,919 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:30,919 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,919 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,919 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,919 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:30,919 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:30,920 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,920 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,921 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,921 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:30,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f52c244[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:30,922 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:30,924 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,924 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:30,925 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:30,925 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,927 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,927 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,927 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,928 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,928 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:30,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,931 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:30,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,931 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,931 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,932 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,933 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,933 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,933 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,934 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:30,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:30,939 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:30,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,938 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,938 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,939 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,939 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:30,939 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,939 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:30,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,039 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:31,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:31,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:31,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:31,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,054 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,054 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,054 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:31,054 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:31,054 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,054 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,057 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,057 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,057 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,057 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,057 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:31,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,058 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,059 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:31,059 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,059 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,059 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,059 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-11 03:54:31,060 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,061 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,061 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,061 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,061 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,061 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,071 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:31,074 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:31,081 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:31,084 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:31,085 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/55ae21d1f9e04e6383b5f40e534fdfcd as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:31,088 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:31,088 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 03:54:31,089 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/2e9bf19103884e40b4098c407d44b5c4 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:31,093 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:31,093 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4, entries=9, sequenceid=83, filesize=5.1 K 2023-02-11 03:54:31,094 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 35ms, sequenceid=83, compaction requested=true 2023-02-11 03:54:31,094 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:31,094 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,094 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:31,094 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:31,094 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,094 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,094 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,095 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,095 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,096 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,096 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:31,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b9ef49c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:31,101 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:31,101 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:31,108 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,108 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,113 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,113 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:31,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:31,117 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:31,117 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:31,117 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:31,118 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,118 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,122 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,122 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,122 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,123 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,124 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,125 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,125 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,127 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,128 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:31,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:31,128 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,129 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,129 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,129 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,129 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,229 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:31,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:31,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:31,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:31,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,244 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,244 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,244 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:31,244 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:31,245 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,245 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,246 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,246 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,246 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,246 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,247 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,247 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,248 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:31,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:31,250 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,250 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:31,250 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 03:54:31,251 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,266 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,275 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,294 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,298 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,299 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/bfec3f2ebd69433ba9842d84afcdeda9 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,302 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,302 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 03:54:31,303 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/f2ef0bc9d8574a019eaee0c80d83b35d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,307 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,307 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d, entries=5, sequenceid=91, filesize=5.0 K 2023-02-11 03:54:31,308 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 58ms, sequenceid=91, compaction requested=true 2023-02-11 03:54:31,308 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:31,308 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,308 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:31,308 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:31,308 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,308 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,308 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,310 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,310 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,310 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,310 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:31,312 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@212a1856[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:31,312 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:31,312 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:31,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:31,313 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:31,315 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,315 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,315 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,315 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,317 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:31,320 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:31,320 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:31,320 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:31,320 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:31,320 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,320 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,322 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,323 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,323 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,325 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,325 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,325 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,329 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,330 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:31,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,330 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,331 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,331 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,331 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,331 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,331 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,430 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:31,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:31,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:31,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:31,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,441 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,441 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,441 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-11 03:54:31,441 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-11 03:54:31,441 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,441 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,443 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,443 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,443 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,443 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,444 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,444 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,445 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:31,446 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:31,446 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,446 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. started... 2023-02-11 03:54:31,446 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-11 03:54:31,447 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,447 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,448 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,448 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,448 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,448 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,455 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,459 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,467 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,471 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,472 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/7a69b39d708f46c0ab87e94a9476b21e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,475 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,476 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 03:54:31,476 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/2f78179abb1248c7942b86c7ee0decc6 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,480 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,480 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6, entries=5, sequenceid=99, filesize=5.0 K 2023-02-11 03:54:31,480 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 34ms, sequenceid=99, compaction requested=true 2023-02-11 03:54:31,480 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 99a53710f43740e3cff337d6c3cb3857: 2023-02-11 03:54:31,480 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-11 03:54:31,481 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-11 03:54:31,482 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,482 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,483 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-11 03:54:31,484 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cf28fd7[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:31,484 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-11 03:54:31,484 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,486 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:54:31,486 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,487 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:31,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:54:31,487 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:54:31,488 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,488 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,488 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,489 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,489 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-11 03:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:54:31,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:54:31,492 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-11 03:54:31,492 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:31,492 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:31,492 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,492 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,493 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,494 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,494 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,496 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,496 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,496 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,500 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,500 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-11 03:54:31,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:31,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:31,501 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:54:31,502 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:31,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,602 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:31,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:31,605 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,606 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:31,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:31,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:31,607 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:31,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, UNASSIGN}] 2023-02-11 03:54:31,609 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, UNASSIGN 2023-02-11 03:54:31,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=99a53710f43740e3cff337d6c3cb3857, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:31,610 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087671609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087671609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087671609"}]},"ts":"1676087671609"} 2023-02-11 03:54:31,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 99a53710f43740e3cff337d6c3cb3857, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:31,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 99a53710f43740e3cff337d6c3cb3857, disabling compactions & flushes 2023-02-11 03:54:31,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. after waiting 0 ms 2023-02-11 03:54:31,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 99a53710f43740e3cff337d6c3cb3857 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-11 03:54:31,774 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,781 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,806 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,806 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf1/03a3243b4eba42ce819fea4a7445bff5 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,811 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 03:54:31,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.tmp/cf2/14d31ecaa6c044dc95193d6813ead98d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d, entries=1, sequenceid=103, filesize=4.9 K 2023-02-11 03:54:31,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 54ms, sequenceid=103, compaction requested=true 2023-02-11 03:54:31,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-11 03:54:31,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:31,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. 2023-02-11 03:54:31,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 99a53710f43740e3cff337d6c3cb3857: Waiting for close lock at 1676087671763Running coprocessor pre-close hooks at 1676087671763Disabling compacts and flushes for region at 1676087671763Disabling writes for close at 1676087671763Obtaining lock to block concurrent updates at 1676087671763Preparing flush snapshotting stores in 99a53710f43740e3cff337d6c3cb3857 at 1676087671763Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676087671763Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857. at 1676087671764 (+1 ms)Flushing 99a53710f43740e3cff337d6c3cb3857/cf1: creating writer at 1676087671764Flushing 99a53710f43740e3cff337d6c3cb3857/cf1: appending metadata at 1676087671766 (+2 ms)Flushing 99a53710f43740e3cff337d6c3cb3857/cf1: closing flushed file at 1676087671766Flushing 99a53710f43740e3cff337d6c3cb3857/cf2: creating writer at 1676087671781 (+15 ms)Flushing 99a53710f43740e3cff337d6c3cb3857/cf2: appending metadata at 1676087671784 (+3 ms)Flushing 99a53710f43740e3cff337d6c3cb3857/cf2: closing flushed file at 1676087671784Flushing 99a53710f43740e3cff337d6c3cb3857/cf1: reopening flushed file at 1676087671807 (+23 ms)Flushing 99a53710f43740e3cff337d6c3cb3857/cf2: reopening flushed file at 1676087671813 (+6 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 99a53710f43740e3cff337d6c3cb3857 in 54ms, sequenceid=103, compaction requested=true at 1676087671817 (+4 ms)Writing region close event to WAL at 1676087671821 (+4 ms)Running coprocessor post-close hooks at 1676087671824 (+3 ms)Closed at 1676087671825 (+1 ms) 2023-02-11 03:54:31,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,827 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=99a53710f43740e3cff337d6c3cb3857, regionState=CLOSED 2023-02-11 03:54:31,827 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087671827"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087671827"}]},"ts":"1676087671827"} 2023-02-11 03:54:31,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-11 03:54:31,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 99a53710f43740e3cff337d6c3cb3857, server=jenkins-hbase4.apache.org,36661,1676087644839 in 217 msec 2023-02-11 03:54:31,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-11 03:54:31,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=99a53710f43740e3cff337d6c3cb3857, UNASSIGN in 221 msec 2023-02-11 03:54:31,845 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=99a53710f43740e3cff337d6c3cb3857, threads=16 2023-02-11 03:54:31,848 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,850 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,850 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,850 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,850 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,850 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,851 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,852 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,854 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,855 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,856 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,860 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,865 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704, top=false 2023-02-11 03:54:31,866 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c, top=false 2023-02-11 03:54:31,867 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:31,871 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,872 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a, top=false 2023-02-11 03:54:31,872 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,872 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1, top=false 2023-02-11 03:54:31,876 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,883 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:31,888 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704 for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,891 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,891 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,895 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,897 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:31,897 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d, top=true 2023-02-11 03:54:31,897 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:31,897 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d, top=true 2023-02-11 03:54:31,900 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:31,900 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e, top=true 2023-02-11 03:54:31,916 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:31,916 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6, top=true 2023-02-11 03:54:31,917 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,921 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,921 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,924 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:31,924 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5, top=true 2023-02-11 03:54:31,940 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,943 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:31,943 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9, top=true 2023-02-11 03:54:31,972 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,973 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1 for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,986 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6 for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,986 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,986 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9 for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,987 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,993 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5 for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,994 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,994 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d for child: 84b406d49c28996244aa26bd64f0d4db, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:31,994 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,003 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,004 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,011 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d, top=false 2023-02-11 03:54:32,012 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc, top=false 2023-02-11 03:54:32,037 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:32,044 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:32,069 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,076 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/.splits/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d for child: 9a4543f81593d1f567e1094bf15375ba, parent: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,082 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,085 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,085 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,086 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,100 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,112 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,112 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,116 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,116 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,116 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 for region: 99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,116 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 99a53710f43740e3cff337d6c3cb3857 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:54:32,159 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 03:54:32,160 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-11 03:54:32,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676087672162"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676087672162"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676087672162"}]},"ts":"1676087672162"} 2023-02-11 03:54:32,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672162"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087672162"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087672162"}]},"ts":"1676087672162"} 2023-02-11 03:54:32,162 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672162"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087672162"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087672162"}]},"ts":"1676087672162"} 2023-02-11 03:54:32,165 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, ASSIGN}] 2023-02-11 03:54:32,166 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, ASSIGN 2023-02-11 03:54:32,167 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:32,167 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, ASSIGN 2023-02-11 03:54:32,168 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:54:32,317 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:54:32,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=84b406d49c28996244aa26bd64f0d4db, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:32,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672318"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087672318"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087672318"}]},"ts":"1676087672318"} 2023-02-11 03:54:32,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 84b406d49c28996244aa26bd64f0d4db, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:32,322 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=9a4543f81593d1f567e1094bf15375ba, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:32,322 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672322"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087672322"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087672322"}]},"ts":"1676087672322"} 2023-02-11 03:54:32,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure 9a4543f81593d1f567e1094bf15375ba, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:32,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 84b406d49c28996244aa26bd64f0d4db, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.', STARTKEY => '60', ENDKEY => ''} 2023-02-11 03:54:32,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:32,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:32,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,478 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,479 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1 2023-02-11 03:54:32,479 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1 2023-02-11 03:54:32,479 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b406d49c28996244aa26bd64f0d4db columnFamilyName cf1 2023-02-11 03:54:32,489 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-top 2023-02-11 03:54:32,492 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,493 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-top 2023-02-11 03:54:32,496 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-top 2023-02-11 03:54:32,499 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-top 2023-02-11 03:54:32,502 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:32,502 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:32,505 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:32,505 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:32,508 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:32,508 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:32,508 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(310): Store=84b406d49c28996244aa26bd64f0d4db/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:32,508 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,509 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2 2023-02-11 03:54:32,509 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2 2023-02-11 03:54:32,509 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 84b406d49c28996244aa26bd64f0d4db columnFamilyName cf2 2023-02-11 03:54:32,518 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,519 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-top 2023-02-11 03:54:32,523 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-top 2023-02-11 03:54:32,526 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-top 2023-02-11 03:54:32,529 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-top 2023-02-11 03:54:32,532 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:32,532 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:32,535 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:32,535 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:32,538 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:32,538 DEBUG [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:32,538 INFO [StoreOpener-84b406d49c28996244aa26bd64f0d4db-1] regionserver.HStore(310): Store=84b406d49c28996244aa26bd64f0d4db/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:32,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:32,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:32,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 84b406d49c28996244aa26bd64f0d4db; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60456943, jitterRate=-0.0991213470697403}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:32,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 84b406d49c28996244aa26bd64f0d4db: Running coprocessor pre-open hook at 1676087672476Writing region info on filesystem at 1676087672476Initializing all the Stores at 1676087672477 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087672477Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087672478 (+1 ms)Cleaning up temporary data from old regions at 1676087672540 (+62 ms)Cleaning up detritus from prior splits at 1676087672540Running coprocessor post-open hooks at 1676087672543 (+3 ms)Region opened successfully at 1676087672543 2023-02-11 03:54:32,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db., pid=77, masterSystemTime=1676087672472 2023-02-11 03:54:32,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:32,545 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:32,545 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:32,545 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,545 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,545 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): 84b406d49c28996244aa26bd64f0d4db/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:32,545 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): 84b406d49c28996244aa26bd64f0d4db/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:32,545 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 84b406d49c28996244aa26bd64f0d4db/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,545 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 84b406d49c28996244aa26bd64f0d4db/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,546 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp, totalSize=34.8 K 2023-02-11 03:54:32,546 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp, totalSize=34.8 K 2023-02-11 03:54:32,546 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087669916 2023-02-11 03:54:32,546 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087669916 2023-02-11 03:54:32,547 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087670091 2023-02-11 03:54:32,547 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087670091 2023-02-11 03:54:32,547 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087670289 2023-02-11 03:54:32,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,547 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087670289 2023-02-11 03:54:32,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9a4543f81593d1f567e1094bf15375ba, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.', STARTKEY => '', ENDKEY => '60'} 2023-02-11 03:54:32,547 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676087671041 2023-02-11 03:54:32,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=84b406d49c28996244aa26bd64f0d4db, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:32,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672548"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087672548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087672548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087672548"}]},"ts":"1676087672548"} 2023-02-11 03:54:32,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:32,548 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676087671041 2023-02-11 03:54:32,548 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:32,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,549 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,549 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,549 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,550 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,550 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-11 03:54:32,552 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 84b406d49c28996244aa26bd64f0d4db, server=jenkins-hbase4.apache.org,36661,1676087644839 in 229 msec 2023-02-11 03:54:32,554 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1 2023-02-11 03:54:32,554 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1 2023-02-11 03:54:32,554 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a4543f81593d1f567e1094bf15375ba columnFamilyName cf1 2023-02-11 03:54:32,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, ASSIGN in 387 msec 2023-02-11 03:54:32,570 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 84b406d49c28996244aa26bd64f0d4db#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,571 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 84b406d49c28996244aa26bd64f0d4db#cf1#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,575 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-bottom 2023-02-11 03:54:32,576 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,576 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-bottom 2023-02-11 03:54:32,579 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-bottom 2023-02-11 03:54:32,582 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-bottom 2023-02-11 03:54:32,585 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1 2023-02-11 03:54:32,588 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc 2023-02-11 03:54:32,591 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d 2023-02-11 03:54:32,592 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(310): Store=9a4543f81593d1f567e1094bf15375ba/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:32,592 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,593 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2 2023-02-11 03:54:32,593 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2 2023-02-11 03:54:32,593 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a4543f81593d1f567e1094bf15375ba columnFamilyName cf2 2023-02-11 03:54:32,596 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:32,605 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0dfc83ee1cc94c6cbb1e3159d4e7ce12: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,605 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6bea012d33614acb813563c70a0e5bf1: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,606 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:32,606 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp/cf2/6bea012d33614acb813563c70a0e5bf1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/6bea012d33614acb813563c70a0e5bf1 2023-02-11 03:54:32,606 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-bottom 2023-02-11 03:54:32,606 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12 2023-02-11 03:54:32,611 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0dfc83ee1cc94c6cbb1e3159d4e7ce12: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,612 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 84b406d49c28996244aa26bd64f0d4db/cf1 of 84b406d49c28996244aa26bd64f0d4db into 0dfc83ee1cc94c6cbb1e3159d4e7ce12(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,612 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 84b406d49c28996244aa26bd64f0d4db: 2023-02-11 03:54:32,612 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db., storeName=84b406d49c28996244aa26bd64f0d4db/cf1, priority=9, startTime=1676087672544; duration=0sec 2023-02-11 03:54:32,612 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,612 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-bottom 2023-02-11 03:54:32,612 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 6bea012d33614acb813563c70a0e5bf1: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,613 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 84b406d49c28996244aa26bd64f0d4db/cf2 of 84b406d49c28996244aa26bd64f0d4db into 6bea012d33614acb813563c70a0e5bf1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,613 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 84b406d49c28996244aa26bd64f0d4db: 2023-02-11 03:54:32,613 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db., storeName=84b406d49c28996244aa26bd64f0d4db/cf2, priority=9, startTime=1676087672545; duration=0sec 2023-02-11 03:54:32,613 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,620 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-bottom 2023-02-11 03:54:32,623 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-bottom 2023-02-11 03:54:32,626 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a 2023-02-11 03:54:32,630 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c 2023-02-11 03:54:32,634 DEBUG [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704 2023-02-11 03:54:32,634 INFO [StoreOpener-9a4543f81593d1f567e1094bf15375ba-1] regionserver.HStore(310): Store=9a4543f81593d1f567e1094bf15375ba/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:32,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:32,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:32,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9a4543f81593d1f567e1094bf15375ba; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74696673, jitterRate=0.11306716501712799}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:32,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9a4543f81593d1f567e1094bf15375ba: Running coprocessor pre-open hook at 1676087672549Writing region info on filesystem at 1676087672549Initializing all the Stores at 1676087672551 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087672551Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087672552 (+1 ms)Cleaning up temporary data from old regions at 1676087672636 (+84 ms)Cleaning up detritus from prior splits at 1676087672637 (+1 ms)Running coprocessor post-open hooks at 1676087672640 (+3 ms)Region opened successfully at 1676087672640 2023-02-11 03:54:32,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba., pid=78, masterSystemTime=1676087672472 2023-02-11 03:54:32,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,642 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:32,642 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:54:32,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:32,642 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,642 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,642 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): 9a4543f81593d1f567e1094bf15375ba/cf1 is initiating minor compaction (all files) 2023-02-11 03:54:32,642 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): 9a4543f81593d1f567e1094bf15375ba/cf2 is initiating minor compaction (all files) 2023-02-11 03:54:32,643 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a4543f81593d1f567e1094bf15375ba/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,643 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a4543f81593d1f567e1094bf15375ba/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,643 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp, totalSize=35.0 K 2023-02-11 03:54:32,643 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp, totalSize=35.0 K 2023-02-11 03:54:32,643 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087669916 2023-02-11 03:54:32,643 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087669916 2023-02-11 03:54:32,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,644 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087670091 2023-02-11 03:54:32,644 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087670091 2023-02-11 03:54:32,644 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087670289 2023-02-11 03:54:32,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=9a4543f81593d1f567e1094bf15375ba, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087670289 2023-02-11 03:54:32,645 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087672644"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087672644"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087672644"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087672644"}]},"ts":"1676087672644"} 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087670475 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087670475 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087670676 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087670676 2023-02-11 03:54:32,645 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087670867 2023-02-11 03:54:32,646 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087670867 2023-02-11 03:54:32,646 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087671041 2023-02-11 03:54:32,646 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087671041 2023-02-11 03:54:32,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-11 03:54:32,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure 9a4543f81593d1f567e1094bf15375ba, server=jenkins-hbase4.apache.org,36661,1676087644839 in 323 msec 2023-02-11 03:54:32,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-11 03:54:32,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, ASSIGN in 484 msec 2023-02-11 03:54:32,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db in 1.0450 sec 2023-02-11 03:54:32,660 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 9a4543f81593d1f567e1094bf15375ba#cf2#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,661 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9a4543f81593d1f567e1094bf15375ba#cf1#compaction#75 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,676 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp/cf2/71165b58e4764723956800533ca08fa6 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/71165b58e4764723956800533ca08fa6 2023-02-11 03:54:32,677 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp/cf1/19076e6e3fca4b0b9fd81fa852e5cefe as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/19076e6e3fca4b0b9fd81fa852e5cefe 2023-02-11 03:54:32,681 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9a4543f81593d1f567e1094bf15375ba/cf2 of 9a4543f81593d1f567e1094bf15375ba into 71165b58e4764723956800533ca08fa6(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,681 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a4543f81593d1f567e1094bf15375ba: 2023-02-11 03:54:32,681 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba., storeName=9a4543f81593d1f567e1094bf15375ba/cf2, priority=9, startTime=1676087672642; duration=0sec 2023-02-11 03:54:32,681 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9a4543f81593d1f567e1094bf15375ba/cf1 of 9a4543f81593d1f567e1094bf15375ba into 19076e6e3fca4b0b9fd81fa852e5cefe(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,681 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,681 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a4543f81593d1f567e1094bf15375ba: 2023-02-11 03:54:32,681 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba., storeName=9a4543f81593d1f567e1094bf15375ba/cf1, priority=9, startTime=1676087672641; duration=0sec 2023-02-11 03:54:32,681 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,866 DEBUG [Listener at localhost/45015] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:54:32,869 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:54:32,869 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.HStore(1912): 9a4543f81593d1f567e1094bf15375ba/cf1 is initiating major compaction (all files) 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3841840c; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,870 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a4543f81593d1f567e1094bf15375ba/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:54:32,870 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/19076e6e3fca4b0b9fd81fa852e5cefe] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp, totalSize=6.8 K 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.HStore(1912): 9a4543f81593d1f567e1094bf15375ba/cf2 is initiating major compaction (all files) 2023-02-11 03:54:32,870 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@e73d2e2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,871 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9a4543f81593d1f567e1094bf15375ba/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:32,871 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/71165b58e4764723956800533ca08fa6] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp, totalSize=6.8 K 2023-02-11 03:54:32,871 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 19076e6e3fca4b0b9fd81fa852e5cefe, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087669916 2023-02-11 03:54:32,871 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 71165b58e4764723956800533ca08fa6, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087669916 2023-02-11 03:54:32,872 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,872 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:54:32,872 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.HStore(1912): 84b406d49c28996244aa26bd64f0d4db/cf1 is initiating major compaction (all files) 2023-02-11 03:54:32,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4ca1d325; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:32,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:54:32,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.HStore(1912): 84b406d49c28996244aa26bd64f0d4db/cf2 is initiating major compaction (all files) 2023-02-11 03:54:32,873 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6e279766; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 03:54:32,873 INFO [Listener at localhost/45015] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 03:54:32,875 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 9a4543f81593d1f567e1094bf15375ba#cf2#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,875 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9a4543f81593d1f567e1094bf15375ba#cf1#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,887 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. is not mergeable because it has references 2023-02-11 03:54:32,888 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. is not mergeable because it has references 2023-02-11 03:54:32,890 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp/cf2/7ff2b84be00e4269b17018d4a5957325 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/7ff2b84be00e4269b17018d4a5957325 2023-02-11 03:54:32,890 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/.tmp/cf1/94cf099972724bdfbccdbe728c6012bc as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/94cf099972724bdfbccdbe728c6012bc 2023-02-11 03:54:32,895 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 9a4543f81593d1f567e1094bf15375ba/cf2 of 9a4543f81593d1f567e1094bf15375ba into 7ff2b84be00e4269b17018d4a5957325(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,895 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a4543f81593d1f567e1094bf15375ba: 2023-02-11 03:54:32,896 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 9a4543f81593d1f567e1094bf15375ba/cf1 of 9a4543f81593d1f567e1094bf15375ba into 94cf099972724bdfbccdbe728c6012bc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,896 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9a4543f81593d1f567e1094bf15375ba: 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676087672870; duration=0sec 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676087672870; duration=0sec 2023-02-11 03:54:32,897 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 03:54:32,897 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 84b406d49c28996244aa26bd64f0d4db/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 84b406d49c28996244aa26bd64f0d4db/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp, totalSize=5.0 K 2023-02-11 03:54:32,897 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/6bea012d33614acb813563c70a0e5bf1] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp, totalSize=5.0 K 2023-02-11 03:54:32,897 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 0dfc83ee1cc94c6cbb1e3159d4e7ce12, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,897 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 6bea012d33614acb813563c70a0e5bf1, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-11 03:54:32,900 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 84b406d49c28996244aa26bd64f0d4db#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,900 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 84b406d49c28996244aa26bd64f0d4db#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:32,919 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd10e186203c4115b65b0c4705414be1: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,919 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp/cf1/fd10e186203c4115b65b0c4705414be1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/fd10e186203c4115b65b0c4705414be1 2023-02-11 03:54:32,921 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 741ac35f80b14a0cbb9b7b9544c0977f: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,921 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/.tmp/cf2/741ac35f80b14a0cbb9b7b9544c0977f as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/741ac35f80b14a0cbb9b7b9544c0977f 2023-02-11 03:54:32,923 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd10e186203c4115b65b0c4705414be1: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,924 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 84b406d49c28996244aa26bd64f0d4db/cf1 of 84b406d49c28996244aa26bd64f0d4db into fd10e186203c4115b65b0c4705414be1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,924 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 84b406d49c28996244aa26bd64f0d4db: 2023-02-11 03:54:32,924 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676087672872; duration=0sec 2023-02-11 03:54:32,924 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,925 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 741ac35f80b14a0cbb9b7b9544c0977f: NONE, but ROW specified in column family configuration 2023-02-11 03:54:32,926 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 84b406d49c28996244aa26bd64f0d4db/cf2 of 84b406d49c28996244aa26bd64f0d4db into 741ac35f80b14a0cbb9b7b9544c0977f(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,926 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 84b406d49c28996244aa26bd64f0d4db: 2023-02-11 03:54:32,926 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676087672873; duration=0sec 2023-02-11 03:54:32,926 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,996 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=36661] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. is not mergeable because it has references 2023-02-11 03:54:32,996 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36661] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. is not mergeable because it has references 2023-02-11 03:54:33,002 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-11 03:54:33,003 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:33,005 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:54:33,008 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087673008"}]},"ts":"1676087673008"} 2023-02-11 03:54:33,009 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-11 03:54:33,011 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-11 03:54:33,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, UNASSIGN}] 2023-02-11 03:54:33,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, UNASSIGN 2023-02-11 03:54:33,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=84b406d49c28996244aa26bd64f0d4db, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:33,013 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087673013"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087673013"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087673013"}]},"ts":"1676087673013"} 2023-02-11 03:54:33,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 84b406d49c28996244aa26bd64f0d4db, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:33,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, UNASSIGN 2023-02-11 03:54:33,016 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=9a4543f81593d1f567e1094bf15375ba, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:54:33,016 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087673016"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087673016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087673016"}]},"ts":"1676087673016"} 2023-02-11 03:54:33,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 9a4543f81593d1f567e1094bf15375ba, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:54:33,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:54:33,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9a4543f81593d1f567e1094bf15375ba, disabling compactions & flushes 2023-02-11 03:54:33,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. after waiting 0 ms 2023-02-11 03:54:33,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:33,166 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/19076e6e3fca4b0b9fd81fa852e5cefe, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-bottom] to archive 2023-02-11 03:54:33,167 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:33,168 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,169 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,171 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,172 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-505212d797774861be3ca98bf11d7ce1 2023-02-11 03:54:33,173 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d3aea02c0afc4b20b821c1d6e9917f0d 2023-02-11 03:54:33,174 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bff28646e8ec418dad11d4bad0f7eadc 2023-02-11 03:54:33,175 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/19076e6e3fca4b0b9fd81fa852e5cefe to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/19076e6e3fca4b0b9fd81fa852e5cefe 2023-02-11 03:54:33,176 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,177 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/71165b58e4764723956800533ca08fa6, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-bottom] to archive 2023-02-11 03:54:33,177 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:33,179 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,180 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,181 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,182 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-e13122d44e9947c7a14e8cb2b59e1704 2023-02-11 03:54:33,183 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-d51a0d4ff95b4a0292db24d38e56244a 2023-02-11 03:54:33,184 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-dd0d908cd73d4c36971d8e66a7e0965c 2023-02-11 03:54:33,185 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/71165b58e4764723956800533ca08fa6 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/71165b58e4764723956800533ca08fa6 2023-02-11 03:54:33,186 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 03:54:33,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:33,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba. 2023-02-11 03:54:33,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9a4543f81593d1f567e1094bf15375ba: Waiting for close lock at 1676087673165Running coprocessor pre-close hooks at 1676087673165Disabling compacts and flushes for region at 1676087673165Disabling writes for close at 1676087673165Writing region close event to WAL at 1676087673187 (+22 ms)Running coprocessor post-close hooks at 1676087673190 (+3 ms)Closed at 1676087673191 (+1 ms) 2023-02-11 03:54:33,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:33,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:33,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 84b406d49c28996244aa26bd64f0d4db, disabling compactions & flushes 2023-02-11 03:54:33,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:33,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:33,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. after waiting 0 ms 2023-02-11 03:54:33,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:33,192 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=9a4543f81593d1f567e1094bf15375ba, regionState=CLOSED 2023-02-11 03:54:33,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087673192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087673192"}]},"ts":"1676087673192"} 2023-02-11 03:54:33,193 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5] to archive 2023-02-11 03:54:33,194 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:33,195 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/7ff2365cf96447a4bb2cb2676f480b94.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-11 03:54:33,196 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 9a4543f81593d1f567e1094bf15375ba, server=jenkins-hbase4.apache.org,36661,1676087644839 in 177 msec 2023-02-11 03:54:33,197 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/8f3c83ff6a3d4eed9599f166b9d7a818.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=9a4543f81593d1f567e1094bf15375ba, UNASSIGN in 185 msec 2023-02-11 03:54:33,198 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/4664e026d35f40f9952b9a69386f05e5.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,199 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/55ae21d1f9e04e6383b5f40e534fdfcd.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,200 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:33,202 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:33,203 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/0dfc83ee1cc94c6cbb1e3159d4e7ce12 2023-02-11 03:54:33,204 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:33,205 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/6bea012d33614acb813563c70a0e5bf1, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d] to archive 2023-02-11 03:54:33,206 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:33,207 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/60e53c5965ac4c0abb7b30017e402044.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,209 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/61fa2d760ffe4a758120753eba86ca03.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,210 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/8c18adfbc43d421a97f2f5cb5613c8a9.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,211 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/2e9bf19103884e40b4098c407d44b5c4.99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,212 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:33,213 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:33,214 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/6bea012d33614acb813563c70a0e5bf1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/6bea012d33614acb813563c70a0e5bf1 2023-02-11 03:54:33,216 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/testSplitTableRegionDeletedRowsDaughter=99a53710f43740e3cff337d6c3cb3857-14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:33,219 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-11 03:54:33,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:33,221 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db. 2023-02-11 03:54:33,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 84b406d49c28996244aa26bd64f0d4db: Waiting for close lock at 1676087673192Running coprocessor pre-close hooks at 1676087673192Disabling compacts and flushes for region at 1676087673192Disabling writes for close at 1676087673192Writing region close event to WAL at 1676087673217 (+25 ms)Running coprocessor post-close hooks at 1676087673220 (+3 ms)Closed at 1676087673221 (+1 ms) 2023-02-11 03:54:33,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:33,223 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=84b406d49c28996244aa26bd64f0d4db, regionState=CLOSED 2023-02-11 03:54:33,223 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676087673223"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087673223"}]},"ts":"1676087673223"} 2023-02-11 03:54:33,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-11 03:54:33,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 84b406d49c28996244aa26bd64f0d4db, server=jenkins-hbase4.apache.org,36661,1676087644839 in 210 msec 2023-02-11 03:54:33,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-11 03:54:33,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=84b406d49c28996244aa26bd64f0d4db, UNASSIGN in 215 msec 2023-02-11 03:54:33,228 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087673228"}]},"ts":"1676087673228"} 2023-02-11 03:54:33,229 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-11 03:54:33,230 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-11 03:54:33,232 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 226 msec 2023-02-11 03:54:33,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:54:33,310 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-11 03:54:33,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,313 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,314 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 03:54:33,317 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,317 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:33,317 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:33,319 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/recovered.edits] 2023-02-11 03:54:33,320 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/recovered.edits] 2023-02-11 03:54:33,320 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/recovered.edits] 2023-02-11 03:54:33,322 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/fd10e186203c4115b65b0c4705414be1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf1/fd10e186203c4115b65b0c4705414be1 2023-02-11 03:54:33,322 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/03a3243b4eba42ce819fea4a7445bff5 2023-02-11 03:54:33,322 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/94cf099972724bdfbccdbe728c6012bc to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf1/94cf099972724bdfbccdbe728c6012bc 2023-02-11 03:54:33,324 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/4664e026d35f40f9952b9a69386f05e5 2023-02-11 03:54:33,325 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/741ac35f80b14a0cbb9b7b9544c0977f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/cf2/741ac35f80b14a0cbb9b7b9544c0977f 2023-02-11 03:54:33,325 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/7ff2b84be00e4269b17018d4a5957325 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/cf2/7ff2b84be00e4269b17018d4a5957325 2023-02-11 03:54:33,325 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/505212d797774861be3ca98bf11d7ce1 2023-02-11 03:54:33,326 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/55ae21d1f9e04e6383b5f40e534fdfcd 2023-02-11 03:54:33,327 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/recovered.edits/113.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba/recovered.edits/113.seqid 2023-02-11 03:54:33,327 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/recovered.edits/113.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db/recovered.edits/113.seqid 2023-02-11 03:54:33,327 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7a69b39d708f46c0ab87e94a9476b21e 2023-02-11 03:54:33,328 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/9a4543f81593d1f567e1094bf15375ba 2023-02-11 03:54:33,328 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:54:33,329 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/7ff2365cf96447a4bb2cb2676f480b94 2023-02-11 03:54:33,330 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/8f3c83ff6a3d4eed9599f166b9d7a818 2023-02-11 03:54:33,331 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bfec3f2ebd69433ba9842d84afcdeda9 2023-02-11 03:54:33,332 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/bff28646e8ec418dad11d4bad0f7eadc 2023-02-11 03:54:33,333 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf1/d3aea02c0afc4b20b821c1d6e9917f0d 2023-02-11 03:54:33,335 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/14d31ecaa6c044dc95193d6813ead98d 2023-02-11 03:54:33,337 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2e9bf19103884e40b4098c407d44b5c4 2023-02-11 03:54:33,338 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/2f78179abb1248c7942b86c7ee0decc6 2023-02-11 03:54:33,339 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/60e53c5965ac4c0abb7b30017e402044 2023-02-11 03:54:33,340 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/61fa2d760ffe4a758120753eba86ca03 2023-02-11 03:54:33,341 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/8c18adfbc43d421a97f2f5cb5613c8a9 2023-02-11 03:54:33,342 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/d51a0d4ff95b4a0292db24d38e56244a 2023-02-11 03:54:33,343 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/dd0d908cd73d4c36971d8e66a7e0965c 2023-02-11 03:54:33,344 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/e13122d44e9947c7a14e8cb2b59e1704 2023-02-11 03:54:33,345 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/cf2/f2ef0bc9d8574a019eaee0c80d83b35d 2023-02-11 03:54:33,348 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/recovered.edits/106.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857/recovered.edits/106.seqid 2023-02-11 03:54:33,348 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/99a53710f43740e3cff337d6c3cb3857 2023-02-11 03:54:33,348 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-11 03:54:33,350 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,352 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-11 03:54:33,354 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-11 03:54:33,355 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,355 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-11 03:54:33,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087673355"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:33,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087673355"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:33,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087673355"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:33,357 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:54:33,357 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 99a53710f43740e3cff337d6c3cb3857, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676087669412.99a53710f43740e3cff337d6c3cb3857.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 9a4543f81593d1f567e1094bf15375ba, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676087671605.9a4543f81593d1f567e1094bf15375ba.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 84b406d49c28996244aa26bd64f0d4db, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676087671605.84b406d49c28996244aa26bd64f0d4db.', STARTKEY => '60', ENDKEY => ''}] 2023-02-11 03:54:33,357 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-11 03:54:33,357 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087673357"}]},"ts":"9223372036854775807"} 2023-02-11 03:54:33,358 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-11 03:54:33,360 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:54:33,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 49 msec 2023-02-11 03:54:33,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-11 03:54:33,415 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-11 03:54:33,429 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=405 (was 401) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:46216 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50550 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:46230 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:60252 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59742 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45906 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50470 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:59708 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:60236 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:60258 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45644 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:51046 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45626 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:45598 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:60312 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2003007161_15 at /127.0.0.1:50730 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=861 (was 859) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=195 (was 202), ProcessCount=168 (was 168), AvailableMemoryMB=3171 (was 3258) 2023-02-11 03:54:33,443 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=405, OpenFileDescriptor=861, MaxFileDescriptor=60000, SystemLoadAverage=195, ProcessCount=168, AvailableMemoryMB=3170 2023-02-11 03:54:33,443 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:54:33,450 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:54:33,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:54:33,451 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:54:33,452 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:33,453 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:33,454 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,454 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff empty. 2023-02-11 03:54:33,455 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,455 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 03:54:33,464 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:33,465 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 26eed2c41af1054f00ad510fa2d3a0ff, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 26eed2c41af1054f00ad510fa2d3a0ff, disabling compactions & flushes 2023-02-11 03:54:33,471 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. after waiting 0 ms 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,471 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,471 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 26eed2c41af1054f00ad510fa2d3a0ff: Waiting for close lock at 1676087673471Disabling compacts and flushes for region at 1676087673471Disabling writes for close at 1676087673471Writing region close event to WAL at 1676087673471Closed at 1676087673471 2023-02-11 03:54:33,473 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:33,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087673474"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087673474"}]},"ts":"1676087673474"} 2023-02-11 03:54:33,475 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:33,475 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:33,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087673475"}]},"ts":"1676087673475"} 2023-02-11 03:54:33,476 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-11 03:54:33,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:33,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:33,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:33,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:33,479 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:54:33,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, ASSIGN}] 2023-02-11 03:54:33,481 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, ASSIGN 2023-02-11 03:54:33,481 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36767,1676087644919; forceNewPlan=false, retain=false 2023-02-11 03:54:33,632 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:33,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:33,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087673634"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087673634"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087673634"}]},"ts":"1676087673634"} 2023-02-11 03:54:33,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919}] 2023-02-11 03:54:33,788 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:33,788 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:54:33,790 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41646, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:54:33,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 26eed2c41af1054f00ad510fa2d3a0ff, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:33,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:33,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:33,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,797 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,798 DEBUG [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:54:33,798 DEBUG [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:54:33,799 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 26eed2c41af1054f00ad510fa2d3a0ff columnFamilyName cf1 2023-02-11 03:54:33,799 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] regionserver.HStore(310): Store=26eed2c41af1054f00ad510fa2d3a0ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:33,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:33,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:33,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 26eed2c41af1054f00ad510fa2d3a0ff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71715640, jitterRate=0.06864631175994873}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:33,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 26eed2c41af1054f00ad510fa2d3a0ff: Running coprocessor pre-open hook at 1676087673796Writing region info on filesystem at 1676087673796Initializing all the Stores at 1676087673797 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087673797Cleaning up temporary data from old regions at 1676087673801 (+4 ms)Cleaning up detritus from prior splits at 1676087673801Running coprocessor post-open hooks at 1676087673806 (+5 ms)Region opened successfully at 1676087673806 2023-02-11 03:54:33,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff., pid=87, masterSystemTime=1676087673788 2023-02-11 03:54:33,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:33,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:33,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087673810"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087673810"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087673810"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087673810"}]},"ts":"1676087673810"} 2023-02-11 03:54:33,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-11 03:54:33,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919 in 176 msec 2023-02-11 03:54:33,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-11 03:54:33,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, ASSIGN in 333 msec 2023-02-11 03:54:33,815 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:33,815 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087673815"}]},"ts":"1676087673815"} 2023-02-11 03:54:33,816 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-11 03:54:33,822 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:33,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 372 msec 2023-02-11 03:54:33,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 03:54:33,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:54:33,983 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 03:54:33,984 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:33,984 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-11 03:54:33,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-11 03:54:33,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, REOPEN/MOVE}] 2023-02-11 03:54:33,999 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, REOPEN/MOVE 2023-02-11 03:54:34,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:34,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087674000"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087674000"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087674000"}]},"ts":"1676087674000"} 2023-02-11 03:54:34,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919}] 2023-02-11 03:54:34,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 26eed2c41af1054f00ad510fa2d3a0ff, disabling compactions & flushes 2023-02-11 03:54:34,154 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. after waiting 0 ms 2023-02-11 03:54:34,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-11 03:54:34,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:54:34,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 26eed2c41af1054f00ad510fa2d3a0ff: Waiting for close lock at 1676087674153Running coprocessor pre-close hooks at 1676087674153Disabling compacts and flushes for region at 1676087674153Disabling writes for close at 1676087674154 (+1 ms)Writing region close event to WAL at 1676087674155 (+1 ms)Running coprocessor post-close hooks at 1676087674158 (+3 ms)Closed at 1676087674158 2023-02-11 03:54:34,158 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 26eed2c41af1054f00ad510fa2d3a0ff to self. 2023-02-11 03:54:34,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=CLOSED 2023-02-11 03:54:34,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087674160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087674160"}]},"ts":"1676087674160"} 2023-02-11 03:54:34,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-11 03:54:34,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919 in 161 msec 2023-02-11 03:54:34,165 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36767,1676087644919; forceNewPlan=false, retain=true 2023-02-11 03:54:34,315 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:54:34,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:34,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087674315"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087674315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087674315"}]},"ts":"1676087674315"} 2023-02-11 03:54:34,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919}] 2023-02-11 03:54:34,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 26eed2c41af1054f00ad510fa2d3a0ff, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:34,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:34,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:34,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,473 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,473 DEBUG [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:54:34,473 DEBUG [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:54:34,474 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 26eed2c41af1054f00ad510fa2d3a0ff columnFamilyName cf1 2023-02-11 03:54:34,474 INFO [StoreOpener-26eed2c41af1054f00ad510fa2d3a0ff-1] regionserver.HStore(310): Store=26eed2c41af1054f00ad510fa2d3a0ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:34,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:54:34,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 26eed2c41af1054f00ad510fa2d3a0ff; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64908312, jitterRate=-0.03279078006744385}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:34,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 26eed2c41af1054f00ad510fa2d3a0ff: Running coprocessor pre-open hook at 1676087674472Writing region info on filesystem at 1676087674472Initializing all the Stores at 1676087674472Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087674472Cleaning up temporary data from old regions at 1676087674476 (+4 ms)Cleaning up detritus from prior splits at 1676087674476Running coprocessor post-open hooks at 1676087674478 (+2 ms)Region opened successfully at 1676087674478 2023-02-11 03:54:34,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff., pid=92, masterSystemTime=1676087674468 2023-02-11 03:54:34,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:54:34,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:54:34,481 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087674480"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087674480"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087674480"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087674480"}]},"ts":"1676087674480"} 2023-02-11 03:54:34,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-11 03:54:34,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919 in 165 msec 2023-02-11 03:54:34,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-11 03:54:34,484 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, REOPEN/MOVE in 485 msec 2023-02-11 03:54:34,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-11 03:54:34,487 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-02-11 03:54:34,489 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, ASSIGN}] 2023-02-11 03:54:34,490 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, ASSIGN 2023-02-11 03:54:34,491 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 03:54:34,641 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:54:34,642 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:54:34,642 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:54:34,642 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:54:34,642 DEBUG [jenkins-hbase4:36397] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-11 03:54:34,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=17b39c708c25b93ac4e8b9f831535539, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:54:34,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087674643"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676087674643"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676087674643"}]},"ts":"1676087674643"} 2023-02-11 03:54:34,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 17b39c708c25b93ac4e8b9f831535539, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:54:34,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:54:34,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 17b39c708c25b93ac4e8b9f831535539, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-11 03:54:34,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:54:34,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:54:34,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:34,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:54:34,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:54:34,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:34,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:35,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,415 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-11 03:54:36,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:36,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:37,977 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:38,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:38,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:39,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:40,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:41,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:42,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:43,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:44,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:45,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:46,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:47,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:48,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:49,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:50,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:51,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:52,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:53,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:54,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:55,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:56,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:57,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:58,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:54:59,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:00,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:01,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:02,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:03,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:04,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:05,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,373 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:55:06,402 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:55:06,438 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:55:06,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:06,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:07,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:08,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:09,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:10,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:11,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:12,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:13,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:14,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:15,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:16,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:17,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:18,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:19,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:20,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:21,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:22,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:23,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:24,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:25,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:26,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:27,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:28,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:29,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:30,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:31,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:32,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:33,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:34,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:35,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:35,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,182 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:36,183 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:36,184 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:36,184 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:36,184 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:36,186 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 26eed2c41af1054f00ad510fa2d3a0ff, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:36,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 exec-time=6 msec 2023-02-11 03:55:36,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-11 03:55:36,526 INFO [StoreOpener-17b39c708c25b93ac4e8b9f831535539-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:55:36,529 DEBUG [StoreOpener-17b39c708c25b93ac4e8b9f831535539-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:55:36,529 DEBUG [StoreOpener-17b39c708c25b93ac4e8b9f831535539-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1 2023-02-11 03:55:36,530 INFO [StoreOpener-17b39c708c25b93ac4e8b9f831535539-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 17b39c708c25b93ac4e8b9f831535539 columnFamilyName cf1 2023-02-11 03:55:36,530 INFO [StoreOpener-17b39c708c25b93ac4e8b9f831535539-1] regionserver.HStore(310): Store=17b39c708c25b93ac4e8b9f831535539/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:36,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 17b39c708c25b93ac4e8b9f831535539; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64005937, jitterRate=-0.046237215399742126}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:55:36,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 17b39c708c25b93ac4e8b9f831535539: Running coprocessor pre-open hook at 1676087674805Initializing all the Stores at 1676087736525 (+61720 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087736525Running coprocessor post-open hooks at 1676087736532 (+7 ms)Region opened successfully at 1676087736532 2023-02-11 03:55:36,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539., pid=94, masterSystemTime=1676087674799 2023-02-11 03:55:36,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:36,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:36,534 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=17b39c708c25b93ac4e8b9f831535539, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:36,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087736534"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676087736534"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676087736534"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676087736534"}]},"ts":"1676087736534"} 2023-02-11 03:55:36,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-11 03:55:36,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 17b39c708c25b93ac4e8b9f831535539, server=jenkins-hbase4.apache.org,33925,1676087644887 in 1 mins, 1.89 sec 2023-02-11 03:55:36,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-11 03:55:36,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, ASSIGN in 1 mins, 2.048 sec 2023-02-11 03:55:36,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.575 sec 2023-02-11 03:55:36,541 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676087673957","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62584,"client":"172.31.14.131:50682","queuetimems":0,"class":"HMaster"} 2023-02-11 03:55:36,541 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 603, skipping over this response of 3 bytes 2023-02-11 03:55:38,434 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up 2023-02-11 03:55:38,435 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:38,437 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:38,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087738443"}]},"ts":"1676087738443"} 2023-02-11 03:55:38,444 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-11 03:55:38,446 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-11 03:55:38,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, UNASSIGN}] 2023-02-11 03:55:38,447 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, UNASSIGN 2023-02-11 03:55:38,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=17b39c708c25b93ac4e8b9f831535539, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:38,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087738447"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676087738447"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676087738447"}]},"ts":"1676087738447"} 2023-02-11 03:55:38,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 17b39c708c25b93ac4e8b9f831535539, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:38,451 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, UNASSIGN 2023-02-11 03:55:38,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:38,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087738452"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087738452"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087738452"}]},"ts":"1676087738452"} 2023-02-11 03:55:38,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919}] 2023-02-11 03:55:38,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:38,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:55:38,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 17b39c708c25b93ac4e8b9f831535539, disabling compactions & flushes 2023-02-11 03:55:38,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:38,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:38,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. after waiting 0 ms 2023-02-11 03:55:38,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:38,603 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:38,604 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539. 2023-02-11 03:55:38,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 17b39c708c25b93ac4e8b9f831535539: Waiting for close lock at 1676087738601Running coprocessor pre-close hooks at 1676087738601Disabling compacts and flushes for region at 1676087738601Disabling writes for close at 1676087738602 (+1 ms)Writing region close event to WAL at 1676087738603 (+1 ms)Running coprocessor post-close hooks at 1676087738603Closed at 1676087738604 (+1 ms) 2023-02-11 03:55:38,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 17b39c708c25b93ac4e8b9f831535539 2023-02-11 03:55:38,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:55:38,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 26eed2c41af1054f00ad510fa2d3a0ff, disabling compactions & flushes 2023-02-11 03:55:38,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:55:38,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:55:38,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. after waiting 0 ms 2023-02-11 03:55:38,606 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:55:38,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=17b39c708c25b93ac4e8b9f831535539, regionState=CLOSED 2023-02-11 03:55:38,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087738606"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676087738606"}]},"ts":"1676087738606"} 2023-02-11 03:55:38,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-11 03:55:38,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 17b39c708c25b93ac4e8b9f831535539, server=jenkins-hbase4.apache.org,33925,1676087644887 in 159 msec 2023-02-11 03:55:38,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-11 03:55:38,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:38,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539, UNASSIGN in 165 msec 2023-02-11 03:55:38,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff. 2023-02-11 03:55:38,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 26eed2c41af1054f00ad510fa2d3a0ff: Waiting for close lock at 1676087738605Running coprocessor pre-close hooks at 1676087738605Disabling compacts and flushes for region at 1676087738605Disabling writes for close at 1676087738606 (+1 ms)Writing region close event to WAL at 1676087738606Running coprocessor post-close hooks at 1676087738612 (+6 ms)Closed at 1676087738613 (+1 ms) 2023-02-11 03:55:38,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:55:38,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=26eed2c41af1054f00ad510fa2d3a0ff, regionState=CLOSED 2023-02-11 03:55:38,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676087738614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087738614"}]},"ts":"1676087738614"} 2023-02-11 03:55:38,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-11 03:55:38,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 26eed2c41af1054f00ad510fa2d3a0ff, server=jenkins-hbase4.apache.org,36767,1676087644919 in 162 msec 2023-02-11 03:55:38,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-11 03:55:38,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=26eed2c41af1054f00ad510fa2d3a0ff, UNASSIGN in 170 msec 2023-02-11 03:55:38,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087738618"}]},"ts":"1676087738618"} 2023-02-11 03:55:38,620 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-11 03:55:38,621 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-11 03:55:38,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 184 msec 2023-02-11 03:55:38,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:38,743 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-11 03:55:38,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 03:55:38,751 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:55:38,753 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits] 2023-02-11 03:55:38,757 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits/7.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff/recovered.edits/7.seqid 2023-02-11 03:55:38,758 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/26eed2c41af1054f00ad510fa2d3a0ff 2023-02-11 03:55:38,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-11 03:55:38,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,761 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-11 03:55:38,763 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-11 03:55:38,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-11 03:55:38,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087738764"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:38,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087738764"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:38,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-11 03:55:38,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 26eed2c41af1054f00ad510fa2d3a0ff, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450.26eed2c41af1054f00ad510fa2d3a0ff.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 17b39c708c25b93ac4e8b9f831535539, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676087673450_0001.17b39c708c25b93ac4e8b9f831535539.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-11 03:55:38,766 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-11 03:55:38,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087738766"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:38,768 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-11 03:55:38,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:38,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 27 msec 2023-02-11 03:55:38,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-11 03:55:38,847 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-11 03:55:38,861 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=344 (was 405), OpenFileDescriptor=654 (was 861), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=111 (was 195), ProcessCount=168 (was 168), AvailableMemoryMB=2990 (was 3170) 2023-02-11 03:55:38,876 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=345, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=111, ProcessCount=168, AvailableMemoryMB=2990 2023-02-11 03:55:38,877 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:38,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:55:38,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:55:38,882 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:38,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:38,885 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:38,886 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:38,887 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e empty. 2023-02-11 03:55:38,887 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:38,887 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 03:55:38,901 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:38,902 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => ff1f1cdcfcbef65f4f73c6842eac708e, NAME => 'testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:55:38,940 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:38,941 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing ff1f1cdcfcbef65f4f73c6842eac708e, disabling compactions & flushes 2023-02-11 03:55:38,941 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:38,941 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:38,941 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. after waiting 0 ms 2023-02-11 03:55:38,941 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:38,941 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:38,941 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for ff1f1cdcfcbef65f4f73c6842eac708e: Waiting for close lock at 1676087738941Disabling compacts and flushes for region at 1676087738941Disabling writes for close at 1676087738941Writing region close event to WAL at 1676087738941Closed at 1676087738941 2023-02-11 03:55:38,944 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:38,945 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676087738945"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087738945"}]},"ts":"1676087738945"} 2023-02-11 03:55:38,949 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:38,950 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:38,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087738950"}]},"ts":"1676087738950"} 2023-02-11 03:55:38,952 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-11 03:55:38,956 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:55:38,957 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:55:38,957 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:55:38,957 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:55:38,957 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:55:38,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, ASSIGN}] 2023-02-11 03:55:38,960 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, ASSIGN 2023-02-11 03:55:38,960 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:55:39,111 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:55:39,112 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ff1f1cdcfcbef65f4f73c6842eac708e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,112 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676087739112"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087739112"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087739112"}]},"ts":"1676087739112"} 2023-02-11 03:55:39,113 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure ff1f1cdcfcbef65f4f73c6842eac708e, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:39,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ff1f1cdcfcbef65f4f73c6842eac708e, NAME => 'testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:39,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:39,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:39,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,274 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,275 DEBUG [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1 2023-02-11 03:55:39,275 DEBUG [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1 2023-02-11 03:55:39,276 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff1f1cdcfcbef65f4f73c6842eac708e columnFamilyName cf1 2023-02-11 03:55:39,277 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] regionserver.HStore(310): Store=ff1f1cdcfcbef65f4f73c6842eac708e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:39,277 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,278 DEBUG [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2 2023-02-11 03:55:39,278 DEBUG [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2 2023-02-11 03:55:39,279 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff1f1cdcfcbef65f4f73c6842eac708e columnFamilyName cf2 2023-02-11 03:55:39,279 INFO [StoreOpener-ff1f1cdcfcbef65f4f73c6842eac708e-1] regionserver.HStore(310): Store=ff1f1cdcfcbef65f4f73c6842eac708e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:39,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,283 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:39,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:39,286 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:39,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ff1f1cdcfcbef65f4f73c6842eac708e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59250404, jitterRate=-0.11710017919540405}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:39,287 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ff1f1cdcfcbef65f4f73c6842eac708e: Running coprocessor pre-open hook at 1676087739271Writing region info on filesystem at 1676087739271Initializing all the Stores at 1676087739272 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087739272Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087739274 (+2 ms)Cleaning up temporary data from old regions at 1676087739281 (+7 ms)Cleaning up detritus from prior splits at 1676087739282 (+1 ms)Running coprocessor post-open hooks at 1676087739287 (+5 ms)Region opened successfully at 1676087739287 2023-02-11 03:55:39,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e., pid=104, masterSystemTime=1676087739265 2023-02-11 03:55:39,289 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=ff1f1cdcfcbef65f4f73c6842eac708e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676087739290"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087739290"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087739290"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087739290"}]},"ts":"1676087739290"} 2023-02-11 03:55:39,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-11 03:55:39,294 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure ff1f1cdcfcbef65f4f73c6842eac708e, server=jenkins-hbase4.apache.org,33925,1676087644887 in 179 msec 2023-02-11 03:55:39,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-11 03:55:39,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, ASSIGN in 336 msec 2023-02-11 03:55:39,296 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:39,297 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087739296"}]},"ts":"1676087739296"} 2023-02-11 03:55:39,298 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-11 03:55:39,300 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:39,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 419 msec 2023-02-11 03:55:39,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:39,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:39,394 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:39,394 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,394 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:39,394 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:39,395 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,395 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,397 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,397 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,398 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,398 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,399 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,404 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:39,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:39,405 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,405 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:39,406 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:55:39,407 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,407 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,434 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/934ac49af8124ce29b2fd9e1f2118789 2023-02-11 03:55:39,446 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/3ae3555d3cd54c1d89944e088ea91847 2023-02-11 03:55:39,451 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/934ac49af8124ce29b2fd9e1f2118789 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/934ac49af8124ce29b2fd9e1f2118789 2023-02-11 03:55:39,456 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/934ac49af8124ce29b2fd9e1f2118789, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:39,457 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/3ae3555d3cd54c1d89944e088ea91847 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ae3555d3cd54c1d89944e088ea91847 2023-02-11 03:55:39,461 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ae3555d3cd54c1d89944e088ea91847, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:39,463 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 57ms, sequenceid=6, compaction requested=false 2023-02-11 03:55:39,463 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-11 03:55:39,464 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:39,464 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,464 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:39,464 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:39,464 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,464 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,464 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,466 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,466 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,466 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,467 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:39,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@650b36ef[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:39,469 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:39,469 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,470 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,470 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,470 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,471 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:39,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,471 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,473 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,473 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,474 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,474 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,474 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:39,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:39,477 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:39,477 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:39,477 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:39,477 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:39,477 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,479 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,480 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,480 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,482 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,482 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,482 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,485 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:39,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:39,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,485 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,486 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,586 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:39,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:39,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:39,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:39,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:39,600 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:39,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:39,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:39,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,603 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,603 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,604 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,604 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,604 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:39,605 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:39,605 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,605 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:39,605 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:39,606 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,610 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,610 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,610 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,610 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,610 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,614 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/bf70f160852d470e9894e280431d9618 2023-02-11 03:55:39,626 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/01a10157e20649228ab960e75ceda4d8 2023-02-11 03:55:39,630 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/bf70f160852d470e9894e280431d9618 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618 2023-02-11 03:55:39,633 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:39,634 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/01a10157e20649228ab960e75ceda4d8 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8 2023-02-11 03:55:39,637 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:39,638 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 33ms, sequenceid=19, compaction requested=false 2023-02-11 03:55:39,638 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:39,638 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,638 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:39,638 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:39,638 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,638 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,638 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,640 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,640 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,640 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,640 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:39,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9dd14c6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:39,642 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:39,642 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,644 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:39,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:39,645 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,645 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,645 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:39,644 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,646 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:39,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:39,650 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:39,650 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:39,650 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:39,650 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:39,650 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:39,652 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,652 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,653 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,652 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,652 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,653 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,654 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,654 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,656 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,656 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,656 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,659 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,659 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,660 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:39,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,660 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,661 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,761 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:39,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:39,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:39,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:39,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:39,774 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:39,774 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,775 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:39,775 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:39,775 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,775 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,776 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,776 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,776 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,776 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,777 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,777 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,778 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,778 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,778 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,778 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,779 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,779 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:39,779 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,779 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:39,780 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,783 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,783 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,788 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/2cb8837928134ab4880583b240a02581 2023-02-11 03:55:39,807 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/9669218b10214bbcae453bdb54944665 2023-02-11 03:55:39,812 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/2cb8837928134ab4880583b240a02581 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/2cb8837928134ab4880583b240a02581 2023-02-11 03:55:39,817 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/2cb8837928134ab4880583b240a02581, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:39,818 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/9669218b10214bbcae453bdb54944665 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9669218b10214bbcae453bdb54944665 2023-02-11 03:55:39,823 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9669218b10214bbcae453bdb54944665, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:39,824 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 45ms, sequenceid=32, compaction requested=true 2023-02-11 03:55:39,824 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:39,824 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,824 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:39,824 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:39,824 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,824 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,825 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,826 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,827 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,827 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,827 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:39,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78444503[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:39,831 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:39,831 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,832 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,832 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:39,832 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,833 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,833 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:39,833 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,834 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:39,836 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,836 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:39,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:39,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:39,841 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:39,841 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:39,841 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:39,841 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:39,841 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:39,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:39,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,843 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,844 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:39,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,844 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,845 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,846 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:39,846 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:39,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,851 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,851 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:39,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:39,852 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:39,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:39,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:39,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:39,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,853 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,852 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:39,853 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:39,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:39,953 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:39,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:39,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:39,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:39,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:39,967 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:39,967 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,967 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:39,967 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:39,967 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,967 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,969 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,969 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,969 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,969 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:39,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:39,970 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:39,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:39,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:39,971 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:39,971 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,972 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,972 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:39,972 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:39,972 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:39,971 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:39,972 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:39,972 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:39,973 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,974 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,974 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,974 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,974 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:39,974 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:39,980 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/cfdbb02272c64717a17a10801264ce7b 2023-02-11 03:55:39,990 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8 2023-02-11 03:55:39,993 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/cfdbb02272c64717a17a10801264ce7b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/cfdbb02272c64717a17a10801264ce7b 2023-02-11 03:55:39,996 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/cfdbb02272c64717a17a10801264ce7b, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:39,997 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8 2023-02-11 03:55:40,000 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:40,001 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 29ms, sequenceid=45, compaction requested=true 2023-02-11 03:55:40,001 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:40,001 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:40,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:40,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,002 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,003 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,003 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,004 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:40,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3080e1eb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:40,005 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:40,005 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,006 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,006 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,006 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,007 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,007 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:40,007 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,009 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,008 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,009 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,009 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,008 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,009 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:40,012 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:40,012 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:40,012 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:40,012 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:40,012 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:40,012 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,013 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,014 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,014 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,014 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,016 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,016 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,016 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,019 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:40,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:40,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,019 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:40,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,020 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:40,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,020 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,020 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,020 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,020 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,120 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:40,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:40,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:40,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:40,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:40,133 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:40,133 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,133 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:40,133 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:40,133 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,133 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,136 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,137 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,137 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,138 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:40,138 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:40,138 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,138 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:40,139 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:40,140 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,141 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,141 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,141 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,141 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,141 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,146 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/81243f3136664ea9858cd84023b444e0 2023-02-11 03:55:40,156 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/596dbf0d032c4c6aa29f82d43efd99d7 2023-02-11 03:55:40,160 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/81243f3136664ea9858cd84023b444e0 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/81243f3136664ea9858cd84023b444e0 2023-02-11 03:55:40,163 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/81243f3136664ea9858cd84023b444e0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:40,164 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/596dbf0d032c4c6aa29f82d43efd99d7 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/596dbf0d032c4c6aa29f82d43efd99d7 2023-02-11 03:55:40,167 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/596dbf0d032c4c6aa29f82d43efd99d7, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:40,168 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 29ms, sequenceid=58, compaction requested=true 2023-02-11 03:55:40,168 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:40,168 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,168 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:40,168 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:40,168 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,168 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,168 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,170 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,170 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,170 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,170 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:40,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c00055a[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:40,172 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:40,172 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,178 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,187 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,187 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,187 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,187 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:40,190 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:40,190 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:40,190 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:40,190 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:40,190 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:40,190 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,194 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,194 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,196 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,196 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,198 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:40,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:40,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,199 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,199 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,299 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:40,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:40,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:40,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:40,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-11 03:55:40,312 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-11 03:55:40,312 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-11 03:55:40,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-11 03:55:40,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,313 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,314 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,314 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,314 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,314 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,315 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,315 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,316 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,316 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,316 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:40,317 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,317 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. started... 2023-02-11 03:55:40,317 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-11 03:55:40,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-11 03:55:40,317 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,317 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,318 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,320 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,320 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,320 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,320 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,320 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,327 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/b58cea9e865e4760849178493594f44a 2023-02-11 03:55:40,339 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/9489eec634fd47f497a690a95516e288 2023-02-11 03:55:40,343 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/b58cea9e865e4760849178493594f44a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/b58cea9e865e4760849178493594f44a 2023-02-11 03:55:40,347 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/b58cea9e865e4760849178493594f44a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:40,347 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/9489eec634fd47f497a690a95516e288 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9489eec634fd47f497a690a95516e288 2023-02-11 03:55:40,353 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9489eec634fd47f497a690a95516e288, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:40,354 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 37ms, sequenceid=71, compaction requested=true 2023-02-11 03:55:40,354 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:40,354 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-11 03:55:40,354 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-11 03:55:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,356 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,356 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:40,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-11 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d1a4b38[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:40,358 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-11 03:55:40,359 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,360 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:40,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-11 03:55:40,361 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,364 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,364 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,365 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-11 03:55:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,365 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,365 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-11 03:55:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-11 03:55:40,368 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-11 03:55:40,368 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:40,368 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:40,368 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-11 03:55:40,368 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:40,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,370 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,371 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,371 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:40,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,372 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,372 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:40,374 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-11 03:55:40,375 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,375 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,378 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-11 03:55:40,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,379 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:40,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:40,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-11 03:55:40,380 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-11 03:55:40,480 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:40,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:40,489 INFO [Listener at localhost/45015] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,491 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.StoreUtils(129): cannot split hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618 because midkey is the same as first or last row 2023-02-11 03:55:40,491 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.StoreUtils(129): cannot split hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8 because midkey is the same as first or last row 2023-02-11 03:55:40,491 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HRegion(2744): Flushing ff1f1cdcfcbef65f4f73c6842eac708e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:55:40,501 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/3ca23ef9930d467b9ea398355fda348d 2023-02-11 03:55:40,511 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/3ea0049ecd244f2aa549ca811a72f5f8 2023-02-11 03:55:40,514 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf1/3ca23ef9930d467b9ea398355fda348d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/3ca23ef9930d467b9ea398355fda348d 2023-02-11 03:55:40,517 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/3ca23ef9930d467b9ea398355fda348d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:40,518 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/.tmp/cf2/3ea0049ecd244f2aa549ca811a72f5f8 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ea0049ecd244f2aa549ca811a72f5f8 2023-02-11 03:55:40,521 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ea0049ecd244f2aa549ca811a72f5f8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:40,522 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ff1f1cdcfcbef65f4f73c6842eac708e in 31ms, sequenceid=82, compaction requested=true 2023-02-11 03:55:40,523 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.HRegion(2445): Flush status journal for ff1f1cdcfcbef65f4f73c6842eac708e: 2023-02-11 03:55:40,523 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.StoreUtils(129): cannot split hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618 because midkey is the same as first or last row 2023-02-11 03:55:40,523 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33925] regionserver.StoreUtils(129): cannot split hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8 because midkey is the same as first or last row 2023-02-11 03:55:40,523 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,523 DEBUG [Listener at localhost/45015] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-11 03:55:40,524 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:40,525 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-11 03:55:40,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-11 03:55:40,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:40,528 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087740528"}]},"ts":"1676087740528"} 2023-02-11 03:55:40,529 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-11 03:55:40,530 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-11 03:55:40,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, UNASSIGN}] 2023-02-11 03:55:40,531 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, UNASSIGN 2023-02-11 03:55:40,532 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ff1f1cdcfcbef65f4f73c6842eac708e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:40,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676087740532"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087740532"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087740532"}]},"ts":"1676087740532"} 2023-02-11 03:55:40,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure ff1f1cdcfcbef65f4f73c6842eac708e, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:40,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:40,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:40,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ff1f1cdcfcbef65f4f73c6842eac708e, disabling compactions & flushes 2023-02-11 03:55:40,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. after waiting 0 ms 2023-02-11 03:55:40,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:55:40,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:40,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e. 2023-02-11 03:55:40,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ff1f1cdcfcbef65f4f73c6842eac708e: Waiting for close lock at 1676087740685Running coprocessor pre-close hooks at 1676087740685Disabling compacts and flushes for region at 1676087740685Disabling writes for close at 1676087740685Writing region close event to WAL at 1676087740689 (+4 ms)Running coprocessor post-close hooks at 1676087740694 (+5 ms)Closed at 1676087740694 2023-02-11 03:55:40,695 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:40,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=ff1f1cdcfcbef65f4f73c6842eac708e, regionState=CLOSED 2023-02-11 03:55:40,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676087740696"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087740696"}]},"ts":"1676087740696"} 2023-02-11 03:55:40,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-11 03:55:40,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure ff1f1cdcfcbef65f4f73c6842eac708e, server=jenkins-hbase4.apache.org,33925,1676087644887 in 164 msec 2023-02-11 03:55:40,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-11 03:55:40,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=ff1f1cdcfcbef65f4f73c6842eac708e, UNASSIGN in 168 msec 2023-02-11 03:55:40,701 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087740701"}]},"ts":"1676087740701"} 2023-02-11 03:55:40,702 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-11 03:55:40,704 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-11 03:55:40,706 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 180 msec 2023-02-11 03:55:40,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:40,830 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-11 03:55:40,831 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-11 03:55:40,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,833 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,833 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 03:55:40,836 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:40,838 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/recovered.edits] 2023-02-11 03:55:40,841 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/2cb8837928134ab4880583b240a02581 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/2cb8837928134ab4880583b240a02581 2023-02-11 03:55:40,842 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/3ca23ef9930d467b9ea398355fda348d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/3ca23ef9930d467b9ea398355fda348d 2023-02-11 03:55:40,843 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/81243f3136664ea9858cd84023b444e0 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/81243f3136664ea9858cd84023b444e0 2023-02-11 03:55:40,845 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/934ac49af8124ce29b2fd9e1f2118789 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/934ac49af8124ce29b2fd9e1f2118789 2023-02-11 03:55:40,846 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/b58cea9e865e4760849178493594f44a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/b58cea9e865e4760849178493594f44a 2023-02-11 03:55:40,847 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/bf70f160852d470e9894e280431d9618 2023-02-11 03:55:40,848 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/cfdbb02272c64717a17a10801264ce7b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf1/cfdbb02272c64717a17a10801264ce7b 2023-02-11 03:55:40,850 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/01a10157e20649228ab960e75ceda4d8 2023-02-11 03:55:40,851 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ae3555d3cd54c1d89944e088ea91847 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ae3555d3cd54c1d89944e088ea91847 2023-02-11 03:55:40,852 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ea0049ecd244f2aa549ca811a72f5f8 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/3ea0049ecd244f2aa549ca811a72f5f8 2023-02-11 03:55:40,853 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/596dbf0d032c4c6aa29f82d43efd99d7 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/596dbf0d032c4c6aa29f82d43efd99d7 2023-02-11 03:55:40,853 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9489eec634fd47f497a690a95516e288 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9489eec634fd47f497a690a95516e288 2023-02-11 03:55:40,854 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9669218b10214bbcae453bdb54944665 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/9669218b10214bbcae453bdb54944665 2023-02-11 03:55:40,855 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/cf2/ecd8a2a6075848918e0dbc81d1fdb5f8 2023-02-11 03:55:40,857 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e/recovered.edits/85.seqid 2023-02-11 03:55:40,858 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testInvalidSplitKey/ff1f1cdcfcbef65f4f73c6842eac708e 2023-02-11 03:55:40,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-11 03:55:40,859 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,861 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-11 03:55:40,864 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-11 03:55:40,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,865 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-11 03:55:40,865 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087740865"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:40,866 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 03:55:40,866 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ff1f1cdcfcbef65f4f73c6842eac708e, NAME => 'testInvalidSplitKey,,1676087738881.ff1f1cdcfcbef65f4f73c6842eac708e.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 03:55:40,866 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-11 03:55:40,866 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087740866"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:40,867 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-11 03:55:40,869 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 38 msec 2023-02-11 03:55:40,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-11 03:55:40,935 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-11 03:55:40,948 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=367 (was 345) - Thread LEAK? -, OpenFileDescriptor=740 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=111 (was 111), ProcessCount=168 (was 168), AvailableMemoryMB=2982 (was 2990) 2023-02-11 03:55:40,961 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=367, OpenFileDescriptor=740, MaxFileDescriptor=60000, SystemLoadAverage=111, ProcessCount=168, AvailableMemoryMB=2982 2023-02-11 03:55:40,962 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:40,964 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:55:40,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:55:40,966 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:40,967 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:40,968 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:40,969 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:40,970 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f empty. 2023-02-11 03:55:40,970 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:40,970 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 03:55:40,981 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:40,982 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => a92471f5b445365ac10fb86a400fb41f, NAME => 'testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing a92471f5b445365ac10fb86a400fb41f, disabling compactions & flushes 2023-02-11 03:55:40,994 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. after waiting 0 ms 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:40,994 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:40,994 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for a92471f5b445365ac10fb86a400fb41f: Waiting for close lock at 1676087740994Disabling compacts and flushes for region at 1676087740994Disabling writes for close at 1676087740994Writing region close event to WAL at 1676087740994Closed at 1676087740994 2023-02-11 03:55:40,996 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:40,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087740997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087740997"}]},"ts":"1676087740997"} 2023-02-11 03:55:40,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:40,999 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:40,999 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087740999"}]},"ts":"1676087740999"} 2023-02-11 03:55:41,000 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-11 03:55:41,004 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:55:41,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:55:41,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:55:41,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:55:41,005 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:55:41,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, ASSIGN}] 2023-02-11 03:55:41,007 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, ASSIGN 2023-02-11 03:55:41,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:41,157 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:55:41,158 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=a92471f5b445365ac10fb86a400fb41f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,159 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087741158"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087741158"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087741158"}]},"ts":"1676087741158"} 2023-02-11 03:55:41,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:41,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a92471f5b445365ac10fb86a400fb41f, NAME => 'testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:41,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,317 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,319 DEBUG [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1 2023-02-11 03:55:41,319 DEBUG [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1 2023-02-11 03:55:41,319 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a92471f5b445365ac10fb86a400fb41f columnFamilyName cf1 2023-02-11 03:55:41,319 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] regionserver.HStore(310): Store=a92471f5b445365ac10fb86a400fb41f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:41,319 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,320 DEBUG [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2 2023-02-11 03:55:41,321 DEBUG [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2 2023-02-11 03:55:41,321 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a92471f5b445365ac10fb86a400fb41f columnFamilyName cf2 2023-02-11 03:55:41,321 INFO [StoreOpener-a92471f5b445365ac10fb86a400fb41f-1] regionserver.HStore(310): Store=a92471f5b445365ac10fb86a400fb41f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:41,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:41,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:41,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:41,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a92471f5b445365ac10fb86a400fb41f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71359999, jitterRate=0.06334684789180756}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:41,326 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a92471f5b445365ac10fb86a400fb41f: Running coprocessor pre-open hook at 1676087741316Writing region info on filesystem at 1676087741316Initializing all the Stores at 1676087741317 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087741317Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087741317Cleaning up temporary data from old regions at 1676087741323 (+6 ms)Cleaning up detritus from prior splits at 1676087741323Running coprocessor post-open hooks at 1676087741326 (+3 ms)Region opened successfully at 1676087741326 2023-02-11 03:55:41,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f., pid=111, masterSystemTime=1676087741311 2023-02-11 03:55:41,328 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,328 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=a92471f5b445365ac10fb86a400fb41f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087741328"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087741328"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087741328"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087741328"}]},"ts":"1676087741328"} 2023-02-11 03:55:41,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-11 03:55:41,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 in 170 msec 2023-02-11 03:55:41,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-11 03:55:41,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, ASSIGN in 326 msec 2023-02-11 03:55:41,333 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:41,333 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087741333"}]},"ts":"1676087741333"} 2023-02-11 03:55:41,334 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-11 03:55:41,336 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:41,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 371 msec 2023-02-11 03:55:41,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:41,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:41,475 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,475 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,475 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:41,475 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:41,476 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,476 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,477 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,477 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,478 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,478 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,479 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,479 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,480 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:41,481 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:55:41,481 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,484 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,484 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,484 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,484 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,484 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,490 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/e79a8ab1fa2f4a5f985fcab80502d658 2023-02-11 03:55:41,501 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/2e5db5cf9e174e3687c15876b402b293 2023-02-11 03:55:41,505 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/e79a8ab1fa2f4a5f985fcab80502d658 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 2023-02-11 03:55:41,508 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:41,509 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/2e5db5cf9e174e3687c15876b402b293 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 2023-02-11 03:55:41,512 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:41,513 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 32ms, sequenceid=6, compaction requested=false 2023-02-11 03:55:41,513 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,514 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:41,514 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:41,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:41,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,515 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,515 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,516 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,516 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:41,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37ac67c5[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:41,518 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:41,518 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:41,519 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:41,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,519 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:41,520 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,520 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,521 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,522 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,522 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,524 ERROR [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 328ac628-71ce-4e94-9ad2-68da08fa646b) service to localhost/127.0.0.1:39467 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:55:41,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:41,525 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:41,525 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:41,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:41,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:41,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,526 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,527 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,528 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,528 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,528 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,530 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,530 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,530 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:41,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:41,534 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:41,533 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,534 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,534 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,534 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,534 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,634 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:41,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:41,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:41,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:41,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:41,646 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,646 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,646 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:41,646 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:41,646 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,646 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,648 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,648 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,648 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,648 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,648 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,648 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,649 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,649 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,650 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,650 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,651 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,651 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:41,651 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:41,651 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,651 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:41,651 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:41,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,653 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,653 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,653 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,653 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,653 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,661 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/291a6f1b28274bc58c8f9ffb8e912639 2023-02-11 03:55:41,672 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/6d005eb443db4186acf9e2243b57b46a 2023-02-11 03:55:41,675 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/291a6f1b28274bc58c8f9ffb8e912639 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 2023-02-11 03:55:41,678 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:41,679 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/6d005eb443db4186acf9e2243b57b46a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a 2023-02-11 03:55:41,682 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:41,683 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 32ms, sequenceid=19, compaction requested=false 2023-02-11 03:55:41,683 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:41,683 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,683 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:41,683 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:41,683 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,683 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,683 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,687 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,687 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:41,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@945762[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:41,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:41,690 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:41,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,693 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:41,694 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:41,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,695 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,696 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,696 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:41,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:41,699 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:41,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:41,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:41,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,699 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,701 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,702 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,702 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,703 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,704 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,704 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:41,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,708 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:41,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:41,708 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,709 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,809 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:41,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:41,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:41,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:41,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:41,822 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,822 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,822 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:41,822 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:41,822 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,822 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,823 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,823 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,823 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,823 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,824 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,824 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,825 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,825 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,825 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,825 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,825 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,826 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,826 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,826 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:41,826 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:41,827 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,828 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,828 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,828 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,828 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,828 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,834 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/78650aa651384650adfabdc5c40f38ca 2023-02-11 03:55:41,843 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/cd8074a1e0644424acf02b6841980897 2023-02-11 03:55:41,847 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/78650aa651384650adfabdc5c40f38ca as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca 2023-02-11 03:55:41,850 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:41,850 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/cd8074a1e0644424acf02b6841980897 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 2023-02-11 03:55:41,853 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:41,854 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 28ms, sequenceid=32, compaction requested=true 2023-02-11 03:55:41,854 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:41,854 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,854 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,856 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,857 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,857 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,857 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:41,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c32b8ff[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:41,859 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:41,859 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:41,860 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,860 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:41,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:41,860 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,862 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,862 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,863 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:41,865 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:41,865 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:41,865 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:41,865 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:41,865 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,865 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:41,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,868 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,869 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:41,869 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:41,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,871 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,871 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:41,871 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:41,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,880 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:41,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:41,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:41,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,880 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:41,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,881 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:41,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:41,981 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:41,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:41,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:41,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:41,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:41,993 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:41,993 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,993 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:41,993 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:41,994 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,994 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,995 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,995 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,995 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:41,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,995 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:41,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:41,997 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:41,997 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:41,997 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:41,998 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:42,000 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,001 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,005 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/cb9291396f87499988854c01b26fd505 2023-02-11 03:55:42,015 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/e9d8ac9e2766459d99ec3f268cb3fdde 2023-02-11 03:55:42,018 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/cb9291396f87499988854c01b26fd505 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 2023-02-11 03:55:42,021 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:42,022 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/e9d8ac9e2766459d99ec3f268cb3fdde as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde 2023-02-11 03:55:42,025 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:42,026 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 29ms, sequenceid=45, compaction requested=true 2023-02-11 03:55:42,026 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:42,026 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:42,026 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:42,026 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:42,026 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,026 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,026 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,028 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,028 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,028 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,028 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:42,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@334e71ca[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:42,031 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:42,031 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:42,033 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,033 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:42,033 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,036 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,037 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,036 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:42,044 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:42,044 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:42,044 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:42,044 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:42,044 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,044 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,045 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,046 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,047 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,047 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,049 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,049 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,049 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:42,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:42,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,052 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:42,052 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,053 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,153 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:42,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:42,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:42,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:42,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:42,166 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,166 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,168 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,168 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,168 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,168 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,168 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,169 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,169 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,169 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,169 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,169 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:42,170 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,170 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,170 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:42,170 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:42,176 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,177 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,177 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,177 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,177 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,177 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,178 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/148a9e3460f94565aa03e292f6edc07d 2023-02-11 03:55:42,188 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/f71d7b2b81d6418b83016a08a67938bc 2023-02-11 03:55:42,192 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/148a9e3460f94565aa03e292f6edc07d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d 2023-02-11 03:55:42,195 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:42,196 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/f71d7b2b81d6418b83016a08a67938bc as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc 2023-02-11 03:55:42,199 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:42,199 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 29ms, sequenceid=58, compaction requested=true 2023-02-11 03:55:42,199 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:42,199 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:42,199 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:42,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:42,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,200 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,204 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,204 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,205 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,205 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:42,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6788aa29[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:42,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:42,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:42,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:42,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,211 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:42,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:42,214 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:42,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:42,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:42,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,214 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,218 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,219 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,219 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:42,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,223 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:42,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,223 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,224 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,324 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:42,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:42,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:42,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:42,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-11 03:55:42,337 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:42,337 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,337 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-11 03:55:42,337 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-11 03:55:42,338 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,338 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,339 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,339 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:42,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,340 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-11 03:55:42,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,341 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-11 03:55:42,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,341 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,341 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:42,341 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:42,341 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. started... 2023-02-11 03:55:42,342 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,350 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/805315c1634f4590873e32666ad3dd5b 2023-02-11 03:55:42,360 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 2023-02-11 03:55:42,363 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/805315c1634f4590873e32666ad3dd5b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b 2023-02-11 03:55:42,366 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:42,367 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 2023-02-11 03:55:42,370 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:42,371 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 29ms, sequenceid=71, compaction requested=true 2023-02-11 03:55:42,371 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a92471f5b445365ac10fb86a400fb41f: 2023-02-11 03:55:42,371 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:42,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:42,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:42,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-11 03:55:42,371 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-11 03:55:42,372 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,372 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,373 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,373 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-11 03:55:42,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4585cefd[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:42,374 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-11 03:55:42,374 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,377 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:42,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-11 03:55:42,377 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:42,378 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,378 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,378 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-11 03:55:42,379 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,379 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-11 03:55:42,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-11 03:55:42,382 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-11 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-11 03:55:42,382 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:42,383 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:42,384 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,383 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,383 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:42,384 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,384 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,385 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:42,386 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-11 03:55:42,387 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,387 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:42,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-11 03:55:42,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:42,393 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:42,393 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:42,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,394 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-11 03:55:42,494 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:42,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:42,752 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-11 03:55:42,752 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:42,753 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:42,754 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:42,754 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:42,754 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:42,755 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:42,755 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:42,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:42,755 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:43,003 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:43,004 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:43,004 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:43,005 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:43,006 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-11 03:55:43,006 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:43,007 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,008 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,009 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,010 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,011 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,011 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:43,011 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,011 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,011 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,012 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,012 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,012 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:43,012 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,012 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:43,012 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,012 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,012 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,012 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:43,013 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:43,013 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:43,014 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087773014 2023-02-11 03:55:43,015 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:43,015 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,017 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:43,018 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:43,018 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:43,018 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087803018 2023-02-11 03:55:43,018 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087863018 2023-02-11 03:55:43,018 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:55:43,018 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:43,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:43,020 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:43,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:43,268 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,268 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:43,268 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:43,268 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:43,268 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:43,269 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:43,269 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:43,269 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:43,269 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:43,269 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:43,271 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-11 03:55:43,271 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:43,272 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:43,272 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:43,272 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:43,272 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:43,273 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:43,275 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,276 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:43,277 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,277 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,277 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,278 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:43,278 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:43,279 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:43,279 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087773279 2023-02-11 03:55:43,281 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:43,281 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,283 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:43,284 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:43,284 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:43,284 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087803284 2023-02-11 03:55:43,284 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087863284 2023-02-11 03:55:43,284 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:55:43,284 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:43,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN}] 2023-02-11 03:55:43,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:43,286 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,286 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:43,286 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:43,534 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,534 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:43,534 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:43,535 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:43,535 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:43,535 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:43,535 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:43,535 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:43,535 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:43,536 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:43,537 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-11 03:55:43,537 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:43,537 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,538 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,539 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,540 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,541 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,542 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,543 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:43,543 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,543 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:43,543 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,543 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,543 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,543 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:43,544 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:43,544 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:43,546 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087773546 2023-02-11 03:55:43,546 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:43,547 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,549 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:43,550 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:43,550 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:43,550 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087803550 2023-02-11 03:55:43,550 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087863550 2023-02-11 03:55:43,550 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:55:43,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN}] 2023-02-11 03:55:43,550 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:43,552 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:43,553 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:43,553 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087743553"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087743553"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087743553"}]},"ts":"1676087743553"} 2023-02-11 03:55:43,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:43,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a92471f5b445365ac10fb86a400fb41f, disabling compactions & flushes 2023-02-11 03:55:43,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:43,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:43,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. after waiting 0 ms 2023-02-11 03:55:43,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:43,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a92471f5b445365ac10fb86a400fb41f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:55:43,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/4a82153a3cae418986d721d0151925be 2023-02-11 03:55:43,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/82faf4d38124498f820ba148ab26ca6f 2023-02-11 03:55:43,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf1/4a82153a3cae418986d721d0151925be as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be 2023-02-11 03:55:43,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:43,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/.tmp/cf2/82faf4d38124498f820ba148ab26ca6f as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f 2023-02-11 03:55:43,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:43,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 31ms, sequenceid=82, compaction requested=true 2023-02-11 03:55:43,741 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:55:43,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:43,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. 2023-02-11 03:55:43,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a92471f5b445365ac10fb86a400fb41f: Waiting for close lock at 1676087743706Running coprocessor pre-close hooks at 1676087743706Disabling compacts and flushes for region at 1676087743706Disabling writes for close at 1676087743706Obtaining lock to block concurrent updates at 1676087743706Preparing flush snapshotting stores in a92471f5b445365ac10fb86a400fb41f at 1676087743706Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087743707 (+1 ms)Flushing stores of testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. at 1676087743707Flushing a92471f5b445365ac10fb86a400fb41f/cf1: creating writer at 1676087743708 (+1 ms)Flushing a92471f5b445365ac10fb86a400fb41f/cf1: appending metadata at 1676087743710 (+2 ms)Flushing a92471f5b445365ac10fb86a400fb41f/cf1: closing flushed file at 1676087743710Flushing a92471f5b445365ac10fb86a400fb41f/cf2: creating writer at 1676087743719 (+9 ms)Flushing a92471f5b445365ac10fb86a400fb41f/cf2: appending metadata at 1676087743721 (+2 ms)Flushing a92471f5b445365ac10fb86a400fb41f/cf2: closing flushed file at 1676087743721Flushing a92471f5b445365ac10fb86a400fb41f/cf1: reopening flushed file at 1676087743731 (+10 ms)Flushing a92471f5b445365ac10fb86a400fb41f/cf2: reopening flushed file at 1676087743734 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a92471f5b445365ac10fb86a400fb41f in 31ms, sequenceid=82, compaction requested=true at 1676087743738 (+4 ms)Writing region close event to WAL at 1676087743739 (+1 ms)Running coprocessor post-close hooks at 1676087743741 (+2 ms)Closed at 1676087743742 (+1 ms) 2023-02-11 03:55:43,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,745 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED 2023-02-11 03:55:43,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087743745"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087743745"}]},"ts":"1676087743745"} 2023-02-11 03:55:43,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-11 03:55:43,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 in 192 msec 2023-02-11 03:55:43,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-11 03:55:43,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN in 197 msec 2023-02-11 03:55:43,751 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:43,751 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:43,751 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:43,800 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,800 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:43,800 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:43,800 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:43,801 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:43,801 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:43,801 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:43,801 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:43,801 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:43,801 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:43,803 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,804 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,805 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,806 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:43,807 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:43,808 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:43,808 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,809 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,809 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,809 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:43,809 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:43,809 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:43,809 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:43,810 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:43,810 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087773810 2023-02-11 03:55:43,812 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:43,812 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:43,815 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:43,815 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:43,815 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:43,815 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087803815 2023-02-11 03:55:43,815 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087863815 2023-02-11 03:55:43,815 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:55:43,819 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:43,828 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=a92471f5b445365ac10fb86a400fb41f, threads=14 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,828 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,829 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,830 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,932 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,956 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,956 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,967 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,970 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,970 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,970 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,971 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,972 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,973 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,973 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,973 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:43,974 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,163 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:44,327 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,327 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region a92471f5b445365ac10fb86a400fb41f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:55:44,359 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:44,359 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,359 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:44,359 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:44,566 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:44,566 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:44,567 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:44,568 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-11 03:55:44,568 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:44,568 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:44,568 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:44,569 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:44,570 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:44,571 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:44,572 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:44,573 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:44,574 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,574 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,574 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,574 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:44,574 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:44,574 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:55:44,574 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:44,575 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:44,575 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087774575 2023-02-11 03:55:44,577 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:44,577 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:44,579 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:44,580 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:44,580 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:44,580 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087804580 2023-02-11 03:55:44,580 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087864580 2023-02-11 03:55:44,580 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:55:44,588 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=a92471f5b445365ac10fb86a400fb41f, threads=14 2023-02-11 03:55:44,589 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,592 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,592 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,593 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,596 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,596 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,596 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,596 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,596 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,597 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,597 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,597 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,600 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,600 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,683 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,684 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,689 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,712 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,712 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,719 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,728 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,735 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,737 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,737 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,738 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,739 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:44,739 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:45,063 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 for region: a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:45,063 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region a92471f5b445365ac10fb86a400fb41f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:55:45,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:45,099 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:45,100 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:45,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:45,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:45,100 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:45,330 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,330 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:45,330 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:45,331 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:45,331 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:45,331 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:45,331 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:45,331 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:45,331 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:45,332 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:45,333 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:45,334 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,335 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,336 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,337 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:45,338 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,338 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,338 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,338 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,338 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-11 03:55:45,339 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:45,339 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:45,340 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087775340 2023-02-11 03:55:45,341 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:45,341 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,343 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:45,344 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:45,344 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:45,344 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087805344 2023-02-11 03:55:45,344 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087865344 2023-02-11 03:55:45,344 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:55:45,346 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:45,348 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:45,348 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:45,348 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:45,594 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,594 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:45,594 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:45,594 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:45,594 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:45,595 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:45,595 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:45,595 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:45,595 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:45,595 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:45,596 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:45,597 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,598 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,599 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,600 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:45,600 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,600 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,600 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,600 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,601 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,602 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:45,602 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,602 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:45,602 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:45,602 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:45,602 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,602 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,603 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,603 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,603 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,603 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:45,604 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:45,604 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:45,604 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087775604 2023-02-11 03:55:45,606 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,606 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:45,608 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:45,608 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:45,608 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:45,608 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087805608 2023-02-11 03:55:45,609 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087865609 2023-02-11 03:55:45,609 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-11 03:55:45,609 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:45,610 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087745610"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676087745610"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676087745610"}]},"ts":"1676087745610"} 2023-02-11 03:55:45,611 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087745610"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087745610"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087745610"}]},"ts":"1676087745610"} 2023-02-11 03:55:45,611 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087745610"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087745610"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087745610"}]},"ts":"1676087745610"} 2023-02-11 03:55:45,612 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:45,612 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:45,612 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:45,859 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:45,859 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:45,860 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:45,860 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:45,861 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-11 03:55:45,862 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:45,862 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:45,862 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:45,862 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:45,863 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:45,863 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:45,863 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,863 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:45,864 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,864 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:45,864 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,864 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:45,864 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,865 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:45,865 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,866 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:45,866 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:45,866 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,866 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:45,866 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,867 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:45,867 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,867 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:45,867 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,867 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:45,868 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,868 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:45,868 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,869 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,869 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:45,869 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,869 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:45,870 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,870 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,870 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:45,870 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:45,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:45,871 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:45,871 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:45,871 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,871 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,871 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,871 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,872 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:45,872 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-11 03:55:45,872 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:45,873 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:45,879 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087775879 2023-02-11 03:55:45,879 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:45,880 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,883 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:45,883 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:45,884 INFO [Listener at localhost/45015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839, table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:45,884 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=247675ee34d83fc0e14d35bb0131bee0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:45,884 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:45,884 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:45,884 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087805884 2023-02-11 03:55:45,884 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087865884 2023-02-11 03:55:45,884 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-11 03:55:45,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087745884"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676087745884"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676087745884"}]},"ts":"1676087745884"} 2023-02-11 03:55:45,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087745884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087745884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087745884"}]},"ts":"1676087745884"} 2023-02-11 03:55:45,885 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087745884"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087745884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087745884"}]},"ts":"1676087745884"} 2023-02-11 03:55:45,887 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:45,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:45,889 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:45,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:45,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:46,134 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:46,135 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:46,135 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:46,136 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:46,137 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-11 03:55:46,137 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:46,137 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:46,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:46,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:46,138 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:46,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:46,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:46,139 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,140 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,140 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,140 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:46,140 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,140 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,141 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,141 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:46,141 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:46,141 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,142 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,142 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,142 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:46,142 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,142 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,143 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,143 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:46,143 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,143 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,143 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,144 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,144 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:46,144 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,144 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,144 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,145 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,145 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,145 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:46,145 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,146 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:46,146 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,146 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:46,146 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,146 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,146 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,146 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,146 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,147 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 03:55:46,147 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:46,148 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:46,148 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087776148 2023-02-11 03:55:46,148 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:46,150 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,152 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:46,152 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:46,152 INFO [Listener at localhost/45015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839, table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:46,152 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=247675ee34d83fc0e14d35bb0131bee0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:46,152 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:46,152 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:46,152 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087806152 2023-02-11 03:55:46,153 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087866153 2023-02-11 03:55:46,153 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:55:46,153 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:46,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN}] 2023-02-11 03:55:46,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:46,155 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:46,155 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:46,403 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:46,403 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:46,404 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:46,404 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:46,405 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-11 03:55:46,405 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:46,405 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,406 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,407 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,408 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,409 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,409 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:46,409 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,409 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,409 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,410 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:46,411 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,411 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,411 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,411 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,412 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-11 03:55:46,412 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:46,413 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:46,413 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087776413 2023-02-11 03:55:46,413 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:46,414 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,417 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:46,417 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:46,417 INFO [Listener at localhost/45015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839, table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:46,417 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=247675ee34d83fc0e14d35bb0131bee0, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:46,417 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-11 03:55:46,417 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:46,417 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087806417 2023-02-11 03:55:46,417 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087866417 2023-02-11 03:55:46,417 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-11 03:55:46,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN}] 2023-02-11 03:55:46,418 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:46,419 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN 2023-02-11 03:55:46,420 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:46,421 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN 2023-02-11 03:55:46,421 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:46,525 ERROR [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 328ac628-71ce-4e94-9ad2-68da08fa646b) service to localhost/127.0.0.1:39467 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:55:46,570 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:55:46,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087746572"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087746572"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087746572"}]},"ts":"1676087746572"} 2023-02-11 03:55:46,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:46,575 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=247675ee34d83fc0e14d35bb0131bee0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,575 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087746575"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087746575"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087746575"}]},"ts":"1676087746575"} 2023-02-11 03:55:46,576 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:46,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b92199dc1a23d92a1aeee9433f6f2ac3, NAME => 'testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 03:55:46,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:46,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:46,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,729 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,730 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1 2023-02-11 03:55:46,730 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1 2023-02-11 03:55:46,730 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b92199dc1a23d92a1aeee9433f6f2ac3 columnFamilyName cf1 2023-02-11 03:55:46,741 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-top 2023-02-11 03:55:46,743 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-top 2023-02-11 03:55:46,746 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-top 2023-02-11 03:55:46,749 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-top 2023-02-11 03:55:46,752 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-top 2023-02-11 03:55:46,755 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-top 2023-02-11 03:55:46,758 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-top 2023-02-11 03:55:46,758 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(310): Store=b92199dc1a23d92a1aeee9433f6f2ac3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:46,758 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,759 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2 2023-02-11 03:55:46,759 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2 2023-02-11 03:55:46,759 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b92199dc1a23d92a1aeee9433f6f2ac3 columnFamilyName cf2 2023-02-11 03:55:46,770 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-top 2023-02-11 03:55:46,773 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-top 2023-02-11 03:55:46,777 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-top 2023-02-11 03:55:46,780 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-top 2023-02-11 03:55:46,783 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-top 2023-02-11 03:55:46,786 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-top 2023-02-11 03:55:46,789 DEBUG [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-top 2023-02-11 03:55:46,789 INFO [StoreOpener-b92199dc1a23d92a1aeee9433f6f2ac3-1] regionserver.HStore(310): Store=b92199dc1a23d92a1aeee9433f6f2ac3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:46,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:46,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b92199dc1a23d92a1aeee9433f6f2ac3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71303255, jitterRate=0.06250129640102386}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:46,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b92199dc1a23d92a1aeee9433f6f2ac3: Running coprocessor pre-open hook at 1676087746728Writing region info on filesystem at 1676087746728Initializing all the Stores at 1676087746729 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087746729Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087746729Cleaning up temporary data from old regions at 1676087746791 (+62 ms)Cleaning up detritus from prior splits at 1676087746792 (+1 ms)Running coprocessor post-open hooks at 1676087746794 (+2 ms)Region opened successfully at 1676087746794 2023-02-11 03:55:46,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3., pid=117, masterSystemTime=1676087746725 2023-02-11 03:55:46,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,797 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:46,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,798 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,798 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:46,798 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): b92199dc1a23d92a1aeee9433f6f2ac3/cf1 is initiating minor compaction (all files) 2023-02-11 03:55:46,798 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,798 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b92199dc1a23d92a1aeee9433f6f2ac3/cf1 in testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,798 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): b92199dc1a23d92a1aeee9433f6f2ac3/cf2 is initiating minor compaction (all files) 2023-02-11 03:55:46,798 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b92199dc1a23d92a1aeee9433f6f2ac3/cf2 in testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,799 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/.tmp, totalSize=34.9 K 2023-02-11 03:55:46,799 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/.tmp, totalSize=34.9 K 2023-02-11 03:55:46,799 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087741469 2023-02-11 03:55:46,799 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087741469 2023-02-11 03:55:46,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:46,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,800 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087741635 2023-02-11 03:55:46,800 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 247675ee34d83fc0e14d35bb0131bee0, NAME => 'testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 03:55:46,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087746800"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087746800"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087746800"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087746800"}]},"ts":"1676087746800"} 2023-02-11 03:55:46,800 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087741635 2023-02-11 03:55:46,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:46,801 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087741810 2023-02-11 03:55:46,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:46,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,801 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087741810 2023-02-11 03:55:46,801 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087741982 2023-02-11 03:55:46,802 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087741982 2023-02-11 03:55:46,802 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087742154 2023-02-11 03:55:46,802 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087742154 2023-02-11 03:55:46,803 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087742325 2023-02-11 03:55:46,803 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087742325 2023-02-11 03:55:46,803 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087742495 2023-02-11 03:55:46,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-11 03:55:46,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839 in 229 msec 2023-02-11 03:55:46,803 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087742495 2023-02-11 03:55:46,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN in 385 msec 2023-02-11 03:55:46,807 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,808 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1 2023-02-11 03:55:46,808 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1 2023-02-11 03:55:46,809 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247675ee34d83fc0e14d35bb0131bee0 columnFamilyName cf1 2023-02-11 03:55:46,814 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): b92199dc1a23d92a1aeee9433f6f2ac3#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:46,819 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): b92199dc1a23d92a1aeee9433f6f2ac3#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:46,821 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-bottom 2023-02-11 03:55:46,824 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-bottom 2023-02-11 03:55:46,827 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-bottom 2023-02-11 03:55:46,830 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-bottom 2023-02-11 03:55:46,837 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-bottom 2023-02-11 03:55:46,843 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/.tmp/cf1/d382680786824def9061153531d7c1e0 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/d382680786824def9061153531d7c1e0 2023-02-11 03:55:46,843 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-bottom 2023-02-11 03:55:46,847 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/.tmp/cf2/513327499bf94f3abfcdee4a1c23830b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/513327499bf94f3abfcdee4a1c23830b 2023-02-11 03:55:46,849 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-bottom 2023-02-11 03:55:46,849 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(310): Store=247675ee34d83fc0e14d35bb0131bee0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:46,849 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,850 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b92199dc1a23d92a1aeee9433f6f2ac3/cf1 of b92199dc1a23d92a1aeee9433f6f2ac3 into d382680786824def9061153531d7c1e0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:46,850 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b92199dc1a23d92a1aeee9433f6f2ac3: 2023-02-11 03:55:46,850 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3., storeName=b92199dc1a23d92a1aeee9433f6f2ac3/cf1, priority=9, startTime=1676087746795; duration=0sec 2023-02-11 03:55:46,850 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,850 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2 2023-02-11 03:55:46,850 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2 2023-02-11 03:55:46,851 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 247675ee34d83fc0e14d35bb0131bee0 columnFamilyName cf2 2023-02-11 03:55:46,851 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b92199dc1a23d92a1aeee9433f6f2ac3/cf2 of b92199dc1a23d92a1aeee9433f6f2ac3 into 513327499bf94f3abfcdee4a1c23830b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:46,851 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b92199dc1a23d92a1aeee9433f6f2ac3: 2023-02-11 03:55:46,851 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3., storeName=b92199dc1a23d92a1aeee9433f6f2ac3/cf2, priority=9, startTime=1676087746797; duration=0sec 2023-02-11 03:55:46,852 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,862 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-bottom 2023-02-11 03:55:46,864 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-bottom 2023-02-11 03:55:46,867 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-bottom 2023-02-11 03:55:46,870 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-bottom 2023-02-11 03:55:46,872 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-bottom 2023-02-11 03:55:46,875 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-bottom 2023-02-11 03:55:46,878 DEBUG [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-bottom 2023-02-11 03:55:46,878 INFO [StoreOpener-247675ee34d83fc0e14d35bb0131bee0-1] regionserver.HStore(310): Store=247675ee34d83fc0e14d35bb0131bee0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:46,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:46,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:46,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 247675ee34d83fc0e14d35bb0131bee0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64501177, jitterRate=-0.03885756433010101}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:46,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 247675ee34d83fc0e14d35bb0131bee0: Running coprocessor pre-open hook at 1676087746801Writing region info on filesystem at 1676087746801Initializing all the Stores at 1676087746802 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087746802Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087746806 (+4 ms)Cleaning up temporary data from old regions at 1676087746881 (+75 ms)Cleaning up detritus from prior splits at 1676087746881Running coprocessor post-open hooks at 1676087746883 (+2 ms)Region opened successfully at 1676087746883 2023-02-11 03:55:46,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0., pid=118, masterSystemTime=1676087746725 2023-02-11 03:55:46,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,885 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:46,885 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:46,885 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:46,885 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,885 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): 247675ee34d83fc0e14d35bb0131bee0/cf1 is initiating minor compaction (all files) 2023-02-11 03:55:46,885 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,885 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 247675ee34d83fc0e14d35bb0131bee0/cf1 in testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,885 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): 247675ee34d83fc0e14d35bb0131bee0/cf2 is initiating minor compaction (all files) 2023-02-11 03:55:46,885 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 247675ee34d83fc0e14d35bb0131bee0/cf2 in testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,886 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/.tmp, totalSize=34.9 K 2023-02-11 03:55:46,886 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/.tmp, totalSize=34.9 K 2023-02-11 03:55:46,886 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087741469 2023-02-11 03:55:46,886 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087741469 2023-02-11 03:55:46,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:46,887 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087741635 2023-02-11 03:55:46,887 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087741635 2023-02-11 03:55:46,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=247675ee34d83fc0e14d35bb0131bee0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087746887"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087746887"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087746887"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087746887"}]},"ts":"1676087746887"} 2023-02-11 03:55:46,887 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087741810 2023-02-11 03:55:46,887 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087741810 2023-02-11 03:55:46,888 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087741982 2023-02-11 03:55:46,888 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087741982 2023-02-11 03:55:46,888 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087742154 2023-02-11 03:55:46,888 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087742154 2023-02-11 03:55:46,889 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087742325 2023-02-11 03:55:46,889 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087742325 2023-02-11 03:55:46,889 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting 4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087742495 2023-02-11 03:55:46,890 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting 82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087742495 2023-02-11 03:55:46,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-11 03:55:46,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839 in 312 msec 2023-02-11 03:55:46,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-11 03:55:46,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN in 473 msec 2023-02-11 03:55:46,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-11 03:55:46,893 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:46,893 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-11 03:55:46,901 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): 247675ee34d83fc0e14d35bb0131bee0#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:46,902 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): 247675ee34d83fc0e14d35bb0131bee0#cf2#compaction#111 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:46,918 INFO [Listener at localhost/45015] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,918 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-11 03:55:46,918 INFO [Listener at localhost/45015] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:46,918 INFO [Listener at localhost/45015] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:46,918 ERROR [Listener at localhost/45015] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-11 03:55:46,918 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/.tmp/cf2/392078a73a194e5e963cbaad94e3211a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/392078a73a194e5e963cbaad94e3211a 2023-02-11 03:55:46,918 INFO [Listener at localhost/45015] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-11 03:55:46,919 INFO [Listener at localhost/45015] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-11 03:55:46,919 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-11 03:55:46,919 INFO [Listener at localhost/45015] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:55:46,919 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:55:46,919 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/.tmp/cf1/c539c1c270544eaf8e27d1a23ee2b3ae as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/c539c1c270544eaf8e27d1a23ee2b3ae 2023-02-11 03:55:46,920 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-11 03:55:46,921 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-11 03:55:46,922 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-11 03:55:46,922 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:55:46,922 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,922 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d8aa82cd827bdff360f5ba5a4ae4262e, daughterA=0f490700d6877b22c5abe6f9e427a663, daughterB=6ecb2d71d524c8ebc2c51ae55e3c1ca2 2023-02-11 03:55:46,923 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,923 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-11 03:55:46,923 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,923 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 247675ee34d83fc0e14d35bb0131bee0/cf2 of 247675ee34d83fc0e14d35bb0131bee0 into 392078a73a194e5e963cbaad94e3211a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:46,923 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=09e96c0d0642358ef90bd77ad1a2435b, daughterA=c21eac229581dfd2abc660b4eaeee1de, daughterB=e400ebe998312232ebe6c6df0af8b6ed 2023-02-11 03:55:46,923 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 247675ee34d83fc0e14d35bb0131bee0: 2023-02-11 03:55:46,924 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0., storeName=247675ee34d83fc0e14d35bb0131bee0/cf2, priority=9, startTime=1676087746884; duration=0sec 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,924 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,924 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 247675ee34d83fc0e14d35bb0131bee0/cf1 of 247675ee34d83fc0e14d35bb0131bee0 into c539c1c270544eaf8e27d1a23ee2b3ae(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:46,924 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 247675ee34d83fc0e14d35bb0131bee0: 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=6a2d161464af91aca78ab4c3203d8177, daughterA=2913b9be4358f3d94713905a911dbe4f, daughterB=92a7bf8419ade3abdc86f26cadcde207 2023-02-11 03:55:46,924 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0., storeName=247675ee34d83fc0e14d35bb0131bee0/cf1, priority=9, startTime=1676087746884; duration=0sec 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=6a2d161464af91aca78ab4c3203d8177, ASSIGN 2023-02-11 03:55:46,924 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:46,924 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=d5f2b752b9b4350ebebfba4a4f344067, daughterA=51be33cfde98666a03236a2e1cc0ce5e, daughterB=d012fc16eb3b4eb095cb578eaf069954 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-11 03:55:46,925 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=99a53710f43740e3cff337d6c3cb3857, daughterA=9a4543f81593d1f567e1094bf15375ba, daughterB=84b406d49c28996244aa26bd64f0d4db 2023-02-11 03:55:46,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-11 03:55:46,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,926 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,33925,1676087644887, table=testRollbackForSplitTableRegionWithReplica, region=17b39c708c25b93ac4e8b9f831535539 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=26eed2c41af1054f00ad510fa2d3a0ff, daughterA=f398f92888363da809a80171efbcf266, daughterB=5936fd0cd0306a5ce59eef109fe33386 2023-02-11 03:55:46,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-11 03:55:46,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,927 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-11 03:55:46,928 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 held the lock before restarting, call acquireLock to restore it. 2023-02-11 03:55:46,929 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,929 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,929 INFO [Listener at localhost/45015] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,929 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a92471f5b445365ac10fb86a400fb41f, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,930 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,930 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,930 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,930 DEBUG [Listener at localhost/45015] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839 didn't hold the lock before restarting, skip acquiring lock. 2023-02-11 03:55:46,930 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-11 03:55:46,930 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-11 03:55:46,931 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887, table=hbase:meta, region=1588230740 2023-02-11 03:55:46,931 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087776931 2023-02-11 03:55:46,932 DEBUG [Listener at localhost/45015] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:55:46,933 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:46,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=af122cb86300c9588537dca293c1ed53, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33925,1676087644887, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887, openSeqNum=2 2023-02-11 03:55:46,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=a92471f5b445365ac10fb86a400fb41f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=2 2023-02-11 03:55:46,935 INFO [Listener at localhost/45015] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36661,1676087644839, table=testRecoveryAndDoubleExecution, region=a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:46,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=247675ee34d83fc0e14d35bb0131bee0, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=86 2023-02-11 03:55:46,935 DEBUG [Listener at localhost/45015] assignment.RegionStateStore(147): Load hbase:meta entry region=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36661,1676087644839, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839, openSeqNum=86 2023-02-11 03:55:46,935 INFO [Listener at localhost/45015] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-11 03:55:46,935 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087806935 2023-02-11 03:55:46,935 INFO [Listener at localhost/45015] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087866935 2023-02-11 03:55:46,936 INFO [Listener at localhost/45015] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-11 03:55:46,936 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-11 03:55:46,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a92471f5b445365ac10fb86a400fb41f, daughterA=247675ee34d83fc0e14d35bb0131bee0, daughterB=b92199dc1a23d92a1aeee9433f6f2ac3 in 4.1830 sec 2023-02-11 03:55:47,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36661] ipc.CallRunner(144): callId: 858 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:49620 deadline: 1676087807186, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f. is not online on jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:47,298 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:55:47,304 INFO [Listener at localhost/45015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 03:55:47,306 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:55:47,314 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:47,315 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-11 03:55:47,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-11 03:55:47,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 03:55:47,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087747318"}]},"ts":"1676087747318"} 2023-02-11 03:55:47,320 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-11 03:55:47,321 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-11 03:55:47,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, UNASSIGN}] 2023-02-11 03:55:47,322 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, UNASSIGN 2023-02-11 03:55:47,323 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:47,323 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087747323"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087747323"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087747323"}]},"ts":"1676087747323"} 2023-02-11 03:55:47,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:47,325 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, UNASSIGN 2023-02-11 03:55:47,326 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=247675ee34d83fc0e14d35bb0131bee0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:47,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087747326"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087747326"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087747326"}]},"ts":"1676087747326"} 2023-02-11 03:55:47,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:47,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 03:55:47,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:47,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b92199dc1a23d92a1aeee9433f6f2ac3, disabling compactions & flushes 2023-02-11 03:55:47,475 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:47,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:47,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. after waiting 0 ms 2023-02-11 03:55:47,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:47,476 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-top] to archive 2023-02-11 03:55:47,477 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:47,478 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,479 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,480 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,481 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,482 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,483 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,484 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,485 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-top] to archive 2023-02-11 03:55:47,486 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:47,487 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,488 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,489 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,490 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,491 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,492 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,493 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:55:47,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:47,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3. 2023-02-11 03:55:47,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b92199dc1a23d92a1aeee9433f6f2ac3: Waiting for close lock at 1676087747475Running coprocessor pre-close hooks at 1676087747475Disabling compacts and flushes for region at 1676087747475Disabling writes for close at 1676087747475Writing region close event to WAL at 1676087747494 (+19 ms)Running coprocessor post-close hooks at 1676087747496 (+2 ms)Closed at 1676087747496 2023-02-11 03:55:47,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:47,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:47,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 247675ee34d83fc0e14d35bb0131bee0, disabling compactions & flushes 2023-02-11 03:55:47,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:47,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:47,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. after waiting 0 ms 2023-02-11 03:55:47,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:47,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b92199dc1a23d92a1aeee9433f6f2ac3, regionState=CLOSED 2023-02-11 03:55:47,499 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087747499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087747499"}]},"ts":"1676087747499"} 2023-02-11 03:55:47,500 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be-bottom] to archive 2023-02-11 03:55:47,500 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:47,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-11 03:55:47,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure b92199dc1a23d92a1aeee9433f6f2ac3, server=jenkins-hbase4.apache.org,36661,1676087644839 in 177 msec 2023-02-11 03:55:47,502 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b92199dc1a23d92a1aeee9433f6f2ac3, UNASSIGN in 180 msec 2023-02-11 03:55:47,503 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,504 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,504 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,505 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,506 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,507 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,508 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f-bottom] to archive 2023-02-11 03:55:47,509 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:47,510 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,510 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,511 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,512 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,513 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,514 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,515 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:55:47,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:47,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0. 2023-02-11 03:55:47,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 247675ee34d83fc0e14d35bb0131bee0: Waiting for close lock at 1676087747498Running coprocessor pre-close hooks at 1676087747498Disabling compacts and flushes for region at 1676087747498Disabling writes for close at 1676087747499 (+1 ms)Writing region close event to WAL at 1676087747515 (+16 ms)Running coprocessor post-close hooks at 1676087747518 (+3 ms)Closed at 1676087747518 2023-02-11 03:55:47,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:47,520 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=247675ee34d83fc0e14d35bb0131bee0, regionState=CLOSED 2023-02-11 03:55:47,521 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087747520"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087747520"}]},"ts":"1676087747520"} 2023-02-11 03:55:47,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-11 03:55:47,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 247675ee34d83fc0e14d35bb0131bee0, server=jenkins-hbase4.apache.org,36661,1676087644839 in 196 msec 2023-02-11 03:55:47,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-11 03:55:47,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=247675ee34d83fc0e14d35bb0131bee0, UNASSIGN in 201 msec 2023-02-11 03:55:47,525 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087747525"}]},"ts":"1676087747525"} 2023-02-11 03:55:47,525 ERROR [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 5cdfedec-baf9-457a-bd3c-897c5ed5a081) service to localhost/127.0.0.1:39467 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:55:47,526 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-11 03:55:47,527 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-11 03:55:47,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 211 msec 2023-02-11 03:55:47,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-11 03:55:47,621 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-11 03:55:47,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-11 03:55:47,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,624 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 03:55:47,628 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,628 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:47,628 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:47,630 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/recovered.edits] 2023-02-11 03:55:47,630 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/recovered.edits] 2023-02-11 03:55:47,630 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/recovered.edits] 2023-02-11 03:55:47,634 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/.regioninfo to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/.regioninfo 2023-02-11 03:55:47,634 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/148a9e3460f94565aa03e292f6edc07d 2023-02-11 03:55:47,634 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/.regioninfo to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/.regioninfo 2023-02-11 03:55:47,635 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/291a6f1b28274bc58c8f9ffb8e912639 2023-02-11 03:55:47,636 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/4a82153a3cae418986d721d0151925be 2023-02-11 03:55:47,637 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,637 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/148a9e3460f94565aa03e292f6edc07d.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,638 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/78650aa651384650adfabdc5c40f38ca 2023-02-11 03:55:47,638 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,638 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/291a6f1b28274bc58c8f9ffb8e912639.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,639 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/805315c1634f4590873e32666ad3dd5b 2023-02-11 03:55:47,639 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,639 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/4a82153a3cae418986d721d0151925be.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,641 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/cb9291396f87499988854c01b26fd505 2023-02-11 03:55:47,641 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,641 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/78650aa651384650adfabdc5c40f38ca.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,642 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf1/e79a8ab1fa2f4a5f985fcab80502d658 2023-02-11 03:55:47,642 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,642 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/805315c1634f4590873e32666ad3dd5b.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,643 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,644 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/cb9291396f87499988854c01b26fd505.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,645 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,645 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/2e5db5cf9e174e3687c15876b402b293 2023-02-11 03:55:47,645 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/e79a8ab1fa2f4a5f985fcab80502d658.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,646 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/6d005eb443db4186acf9e2243b57b46a 2023-02-11 03:55:47,648 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/80461afc5ea54d3e809ac7b3b5ab68f4 2023-02-11 03:55:47,649 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,649 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/2e5db5cf9e174e3687c15876b402b293.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,650 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/82faf4d38124498f820ba148ab26ca6f 2023-02-11 03:55:47,650 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,651 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/6d005eb443db4186acf9e2243b57b46a.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,651 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/cd8074a1e0644424acf02b6841980897 2023-02-11 03:55:47,652 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,652 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/80461afc5ea54d3e809ac7b3b5ab68f4.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,653 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/e9d8ac9e2766459d99ec3f268cb3fdde 2023-02-11 03:55:47,653 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,654 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/82faf4d38124498f820ba148ab26ca6f.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,654 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/cf2/f71d7b2b81d6418b83016a08a67938bc 2023-02-11 03:55:47,655 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,657 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/cd8074a1e0644424acf02b6841980897.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,657 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,658 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/e9d8ac9e2766459d99ec3f268cb3fdde.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,658 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f/recovered.edits/85.seqid 2023-02-11 03:55:47,658 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/247675ee34d83fc0e14d35bb0131bee0/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,659 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,660 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/f71d7b2b81d6418b83016a08a67938bc.a92471f5b445365ac10fb86a400fb41f 2023-02-11 03:55:47,661 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/c539c1c270544eaf8e27d1a23ee2b3ae to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf1/c539c1c270544eaf8e27d1a23ee2b3ae 2023-02-11 03:55:47,662 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/d382680786824def9061153531d7c1e0 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf1/d382680786824def9061153531d7c1e0 2023-02-11 03:55:47,664 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/392078a73a194e5e963cbaad94e3211a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/cf2/392078a73a194e5e963cbaad94e3211a 2023-02-11 03:55:47,664 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/513327499bf94f3abfcdee4a1c23830b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/cf2/513327499bf94f3abfcdee4a1c23830b 2023-02-11 03:55:47,666 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0/recovered.edits/90.seqid 2023-02-11 03:55:47,667 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/247675ee34d83fc0e14d35bb0131bee0 2023-02-11 03:55:47,667 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3/recovered.edits/90.seqid 2023-02-11 03:55:47,668 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testRecoveryAndDoubleExecution/b92199dc1a23d92a1aeee9433f6f2ac3 2023-02-11 03:55:47,668 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-11 03:55:47,670 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,672 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-11 03:55:47,673 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-11 03:55:47,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-11 03:55:47,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087747674"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:47,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087747674"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:47,674 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087747674"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:47,676 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:55:47,676 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a92471f5b445365ac10fb86a400fb41f, NAME => 'testRecoveryAndDoubleExecution,,1676087740965.a92471f5b445365ac10fb86a400fb41f.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 247675ee34d83fc0e14d35bb0131bee0, NAME => 'testRecoveryAndDoubleExecution,,1676087742752.247675ee34d83fc0e14d35bb0131bee0.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b92199dc1a23d92a1aeee9433f6f2ac3, NAME => 'testRecoveryAndDoubleExecution,41,1676087742752.b92199dc1a23d92a1aeee9433f6f2ac3.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 03:55:47,676 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-11 03:55:47,676 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087747676"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:47,677 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-11 03:55:47,679 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-11 03:55:47,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 58 msec 2023-02-11 03:55:47,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-11 03:55:47,725 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-11 03:55:47,740 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=374 (was 367) - Thread LEAK? -, OpenFileDescriptor=842 (was 740) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=94 (was 111), ProcessCount=168 (was 168), AvailableMemoryMB=2942 (was 2982) 2023-02-11 03:55:47,753 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=94, ProcessCount=168, AvailableMemoryMB=2941 2023-02-11 03:55:47,753 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:47,756 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:55:47,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:55:47,758 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:47,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:47,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:47,761 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:47,761 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 empty. 2023-02-11 03:55:47,761 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:47,762 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 03:55:47,770 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:47,771 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3208eb5903dfcc5b74b941b6cef73b91, NAME => 'testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 3208eb5903dfcc5b74b941b6cef73b91, disabling compactions & flushes 2023-02-11 03:55:48,177 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. after waiting 0 ms 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,177 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,177 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 3208eb5903dfcc5b74b941b6cef73b91: Waiting for close lock at 1676087748177Disabling compacts and flushes for region at 1676087748177Disabling writes for close at 1676087748177Writing region close event to WAL at 1676087748177Closed at 1676087748177 2023-02-11 03:55:48,179 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:48,180 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087748180"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087748180"}]},"ts":"1676087748180"} 2023-02-11 03:55:48,181 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:48,182 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:48,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087748182"}]},"ts":"1676087748182"} 2023-02-11 03:55:48,183 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-11 03:55:48,185 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:55:48,186 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:55:48,186 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:55:48,186 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:55:48,186 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:55:48,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, ASSIGN}] 2023-02-11 03:55:48,187 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, ASSIGN 2023-02-11 03:55:48,188 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:48,338 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:55:48,339 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=3208eb5903dfcc5b74b941b6cef73b91, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:48,339 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087748339"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087748339"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087748339"}]},"ts":"1676087748339"} 2023-02-11 03:55:48,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 3208eb5903dfcc5b74b941b6cef73b91, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:48,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3208eb5903dfcc5b74b941b6cef73b91, NAME => 'testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:48,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:48,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:48,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,496 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,497 DEBUG [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1 2023-02-11 03:55:48,497 DEBUG [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1 2023-02-11 03:55:48,497 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3208eb5903dfcc5b74b941b6cef73b91 columnFamilyName cf1 2023-02-11 03:55:48,498 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] regionserver.HStore(310): Store=3208eb5903dfcc5b74b941b6cef73b91/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:48,498 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,499 DEBUG [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2 2023-02-11 03:55:48,499 DEBUG [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2 2023-02-11 03:55:48,499 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3208eb5903dfcc5b74b941b6cef73b91 columnFamilyName cf2 2023-02-11 03:55:48,499 INFO [StoreOpener-3208eb5903dfcc5b74b941b6cef73b91-1] regionserver.HStore(310): Store=3208eb5903dfcc5b74b941b6cef73b91/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:48,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:48,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:48,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:48,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3208eb5903dfcc5b74b941b6cef73b91; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63262823, jitterRate=-0.057310476899147034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:48,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3208eb5903dfcc5b74b941b6cef73b91: Running coprocessor pre-open hook at 1676087748495Writing region info on filesystem at 1676087748495Initializing all the Stores at 1676087748496 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087748496Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087748496Cleaning up temporary data from old regions at 1676087748501 (+5 ms)Cleaning up detritus from prior splits at 1676087748501Running coprocessor post-open hooks at 1676087748504 (+3 ms)Region opened successfully at 1676087748504 2023-02-11 03:55:48,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91., pid=127, masterSystemTime=1676087748492 2023-02-11 03:55:48,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,506 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=3208eb5903dfcc5b74b941b6cef73b91, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:48,506 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087748506"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087748506"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087748506"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087748506"}]},"ts":"1676087748506"} 2023-02-11 03:55:48,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-11 03:55:48,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 3208eb5903dfcc5b74b941b6cef73b91, server=jenkins-hbase4.apache.org,36661,1676087644839 in 166 msec 2023-02-11 03:55:48,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-11 03:55:48,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, ASSIGN in 323 msec 2023-02-11 03:55:48,510 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:48,511 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087748511"}]},"ts":"1676087748511"} 2023-02-11 03:55:48,511 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-11 03:55:48,514 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:48,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 757 msec 2023-02-11 03:55:48,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:48,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,766 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:48,766 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,766 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:48,766 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:48,767 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,767 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,769 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:48,769 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:48,769 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:48,769 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:48,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:48,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:48,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:48,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:48,770 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:48,770 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:48,771 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:48,772 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:48,772 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:48,772 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:48,773 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:55:48,773 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,774 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,774 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,774 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:48,775 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:48,775 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:48,780 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/4334acaa62b64baf8700a969ca0f48ab 2023-02-11 03:55:49,190 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/7d4d8ad9997442cc92e82863bb0e2790 2023-02-11 03:55:49,194 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/4334acaa62b64baf8700a969ca0f48ab as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab 2023-02-11 03:55:49,197 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:49,198 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/7d4d8ad9997442cc92e82863bb0e2790 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790 2023-02-11 03:55:49,201 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:49,201 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 428ms, sequenceid=6, compaction requested=false 2023-02-11 03:55:49,201 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,202 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:49,202 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:49,202 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:49,202 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:49,202 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,202 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,202 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,204 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,204 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,204 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,204 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:49,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ce973f2[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:49,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:49,206 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,209 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,209 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,211 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:49,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:49,214 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:49,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:49,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:49,214 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,214 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,215 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,216 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,216 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,217 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,218 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,218 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,218 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,225 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,225 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:49,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,225 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,226 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,227 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,227 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,243 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:49,326 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:49,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:49,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:49,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:49,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,339 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:49,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:49,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,339 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,340 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,340 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,341 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,341 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,341 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,341 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,342 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,342 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,342 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,343 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,343 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,343 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:49,344 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:49,344 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,355 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/2729682428bf4677be16f525bb80cfda 2023-02-11 03:55:49,766 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/b26bf95effe94bc8a536ee254d6888c5 2023-02-11 03:55:49,770 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/2729682428bf4677be16f525bb80cfda as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda 2023-02-11 03:55:49,773 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:49,774 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/b26bf95effe94bc8a536ee254d6888c5 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5 2023-02-11 03:55:49,777 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:49,777 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 434ms, sequenceid=19, compaction requested=false 2023-02-11 03:55:49,778 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:49,778 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:49,778 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:49,778 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:49,778 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,778 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,778 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,779 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,779 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,780 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,780 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:49,781 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c698984[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:49,781 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:49,781 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,782 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,783 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,783 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:49,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,783 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,784 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,784 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,785 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,786 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,786 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,786 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:49,788 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:49,788 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:49,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:49,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:49,788 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,788 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,790 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,791 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,791 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,792 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,793 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,793 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,796 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,796 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:49,795 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,796 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,797 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,896 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:49,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:49,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:49,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:49,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,908 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,908 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,908 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:49,908 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:49,908 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,908 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,909 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,910 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,909 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,909 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,910 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,910 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,911 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:49,912 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:49,911 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:49,912 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,912 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:49,912 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,912 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:49,912 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,912 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,912 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,916 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,922 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/26cb062853454228a0d5e82a03043d31 2023-02-11 03:55:49,932 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/a3d54398dbe94b2da1e88bd2025b225b 2023-02-11 03:55:49,935 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/26cb062853454228a0d5e82a03043d31 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31 2023-02-11 03:55:49,938 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:49,939 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/a3d54398dbe94b2da1e88bd2025b225b as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b 2023-02-11 03:55:49,942 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:49,942 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 30ms, sequenceid=32, compaction requested=true 2023-02-11 03:55:49,943 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:49,943 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:49,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:49,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:49,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:49,943 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:49,944 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,944 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,945 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:49,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e92a1c2[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:49,946 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:49,946 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,948 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,948 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:49,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:49,948 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:49,953 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,953 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,954 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:49,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:49,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:49,957 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:49,957 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:49,957 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:49,957 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,957 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,958 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:49,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,959 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,959 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,959 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:49,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:49,961 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,961 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,962 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:49,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,965 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:49,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:49,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,965 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:49,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:49,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:49,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:49,966 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:49,967 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,066 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:50,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:50,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:50,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:50,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,078 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:50,078 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,078 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:50,078 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:50,078 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,078 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,080 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,080 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,080 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,080 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,081 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,081 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,081 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:50,082 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:50,082 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:50,082 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:50,082 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:50,083 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,083 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,084 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,090 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/b1ed6b45f3b6472cbaf8ef8932974deb 2023-02-11 03:55:50,500 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/5a89582b37424402b9ab8e151843180d 2023-02-11 03:55:50,504 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/b1ed6b45f3b6472cbaf8ef8932974deb as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb 2023-02-11 03:55:50,507 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:50,508 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/5a89582b37424402b9ab8e151843180d as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d 2023-02-11 03:55:50,511 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:50,512 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 430ms, sequenceid=45, compaction requested=true 2023-02-11 03:55:50,512 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:50,512 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:50,512 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:50,512 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:50,512 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,512 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,512 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,514 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,514 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:50,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@550b62cb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:50,516 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:50,516 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,517 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:50,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:50,517 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,523 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,524 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,524 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:50,526 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:50,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:50,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:50,526 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,526 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,528 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,529 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,529 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,530 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,531 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,531 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:50,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:50,537 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,537 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,538 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,538 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,538 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,538 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,538 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,637 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:50,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:50,648 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:50,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:50,649 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,649 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:50,649 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,649 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:50,649 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:50,649 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,649 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,650 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,650 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,650 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,650 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,651 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,651 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,652 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,653 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,653 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,653 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,653 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:50,653 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:50,653 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:50,653 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:50,653 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:50,654 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,655 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,655 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,655 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,655 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,655 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,661 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/e5edea2a509b4612bab7941a039932bb 2023-02-11 03:55:50,671 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 2023-02-11 03:55:50,674 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/e5edea2a509b4612bab7941a039932bb as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb 2023-02-11 03:55:50,677 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:50,678 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 2023-02-11 03:55:50,681 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:50,682 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 29ms, sequenceid=58, compaction requested=true 2023-02-11 03:55:50,682 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:50,682 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,682 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,684 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,684 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,685 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,685 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:50,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42eaed7b[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:50,686 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:50,686 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:50,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,687 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:50,688 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:50,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,689 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,690 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,690 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:50,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:50,693 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:50,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:50,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:50,693 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,693 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,695 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,696 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,697 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,698 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,699 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:50,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,701 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,702 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,802 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:50,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:50,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:50,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:50,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,814 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:50,814 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,814 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-11 03:55:50,814 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-11 03:55:50,815 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,815 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,816 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,816 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:50,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,816 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:50,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:50,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:50,818 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:50,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-11 03:55:50,818 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. started... 2023-02-11 03:55:50,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:50,818 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:50,818 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:50,819 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:50,821 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:51,226 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/fcbd60dd42224e88a36a71236af23ccf 2023-02-11 03:55:51,236 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/b41e480c394f459691ff2a625d3a7f75 2023-02-11 03:55:51,239 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/fcbd60dd42224e88a36a71236af23ccf as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf 2023-02-11 03:55:51,242 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:51,243 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/b41e480c394f459691ff2a625d3a7f75 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75 2023-02-11 03:55:51,246 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:51,246 INFO [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 428ms, sequenceid=71, compaction requested=true 2023-02-11 03:55:51,247 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 3208eb5903dfcc5b74b941b6cef73b91: 2023-02-11 03:55:51,247 DEBUG [rs(jenkins-hbase4.apache.org,36661,1676087644839)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:51,247 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:51,247 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:51,247 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,247 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-11 03:55:51,247 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-11 03:55:51,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:51,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:51,248 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:51,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,249 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:51,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:51,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:51,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-11 03:55:51,250 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@121f690b[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:51,250 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-11 03:55:51,250 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:51,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:51,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,254 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,254 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:51,254 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:51,255 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:51,255 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:51,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,255 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-11 03:55:51,255 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-11 03:55:51,255 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:51,256 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:51,256 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:51,256 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:51,258 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:51,258 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-11 03:55:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:51,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:51,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36661,1676087644839': 2023-02-11 03:55:51,261 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-11 03:55:51,261 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-11 03:55:51,261 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:51,261 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:51,261 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,261 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:51,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,262 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,263 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,263 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:51,264 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:51,265 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,265 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,266 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:51,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-11 03:55:51,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:51,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:51,268 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:51,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:51,268 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,269 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-11 03:55:51,369 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:51,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:51,378 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,379 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3208eb5903dfcc5b74b941b6cef73b91, daughterA=f7b3b5a8e1cf790554c2a165be1b8efb, daughterB=03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:51,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3208eb5903dfcc5b74b941b6cef73b91, daughterA=f7b3b5a8e1cf790554c2a165be1b8efb, daughterB=03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:51,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3208eb5903dfcc5b74b941b6cef73b91, daughterA=f7b3b5a8e1cf790554c2a165be1b8efb, daughterB=03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:51,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3208eb5903dfcc5b74b941b6cef73b91, daughterA=f7b3b5a8e1cf790554c2a165be1b8efb, daughterB=03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:51,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, UNASSIGN}] 2023-02-11 03:55:51,382 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, UNASSIGN 2023-02-11 03:55:51,383 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=3208eb5903dfcc5b74b941b6cef73b91, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,383 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087751383"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087751383"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087751383"}]},"ts":"1676087751383"} 2023-02-11 03:55:51,384 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 3208eb5903dfcc5b74b941b6cef73b91, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:51,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3208eb5903dfcc5b74b941b6cef73b91, disabling compactions & flushes 2023-02-11 03:55:51,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:51,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:51,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. after waiting 0 ms 2023-02-11 03:55:51,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:51,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 3208eb5903dfcc5b74b941b6cef73b91 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:55:51,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/944593d4aff34b93a54031b2e289eb7f 2023-02-11 03:55:51,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/304ae36ff7644b3bb4df112db76016bd 2023-02-11 03:55:51,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf1/944593d4aff34b93a54031b2e289eb7f as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f 2023-02-11 03:55:51,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:51,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.tmp/cf2/304ae36ff7644b3bb4df112db76016bd as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd 2023-02-11 03:55:51,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:51,566 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 30ms, sequenceid=82, compaction requested=true 2023-02-11 03:55:51,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:55:51,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:51,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. 2023-02-11 03:55:51,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3208eb5903dfcc5b74b941b6cef73b91: Waiting for close lock at 1676087751536Running coprocessor pre-close hooks at 1676087751536Disabling compacts and flushes for region at 1676087751536Disabling writes for close at 1676087751536Obtaining lock to block concurrent updates at 1676087751536Preparing flush snapshotting stores in 3208eb5903dfcc5b74b941b6cef73b91 at 1676087751536Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087751536Flushing stores of testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. at 1676087751537 (+1 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf1: creating writer at 1676087751539 (+2 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf1: appending metadata at 1676087751541 (+2 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf1: closing flushed file at 1676087751541Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf2: creating writer at 1676087751550 (+9 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf2: appending metadata at 1676087751551 (+1 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf2: closing flushed file at 1676087751551Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf1: reopening flushed file at 1676087751560 (+9 ms)Flushing 3208eb5903dfcc5b74b941b6cef73b91/cf2: reopening flushed file at 1676087751563 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3208eb5903dfcc5b74b941b6cef73b91 in 30ms, sequenceid=82, compaction requested=true at 1676087751566 (+3 ms)Writing region close event to WAL at 1676087751568 (+2 ms)Running coprocessor post-close hooks at 1676087751570 (+2 ms)Closed at 1676087751571 (+1 ms) 2023-02-11 03:55:51,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,573 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=3208eb5903dfcc5b74b941b6cef73b91, regionState=CLOSED 2023-02-11 03:55:51,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087751573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087751573"}]},"ts":"1676087751573"} 2023-02-11 03:55:51,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-11 03:55:51,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 3208eb5903dfcc5b74b941b6cef73b91, server=jenkins-hbase4.apache.org,36661,1676087644839 in 190 msec 2023-02-11 03:55:51,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-11 03:55:51,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=3208eb5903dfcc5b74b941b6cef73b91, UNASSIGN in 194 msec 2023-02-11 03:55:51,585 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=3208eb5903dfcc5b74b941b6cef73b91, threads=14 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,585 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,586 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,589 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b, top=false 2023-02-11 03:55:51,590 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb, top=false 2023-02-11 03:55:51,590 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab, top=false 2023-02-11 03:55:51,592 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3, top=false 2023-02-11 03:55:51,593 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,596 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,599 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf, top=false 2023-02-11 03:55:51,599 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75, top=false 2023-02-11 03:55:51,600 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda, top=false 2023-02-11 03:55:51,601 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d, top=false 2023-02-11 03:55:51,602 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb, top=false 2023-02-11 03:55:51,603 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd, top=false 2023-02-11 03:55:51,603 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5, top=false 2023-02-11 03:55:51,604 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790, top=false 2023-02-11 03:55:51,611 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31, top=false 2023-02-11 03:55:51,611 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f, top=false 2023-02-11 03:55:51,616 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3 for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,620 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,624 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75 for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,627 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,632 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,633 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,633 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,634 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790 for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,638 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,638 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,638 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,639 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,640 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,640 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,641 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5 for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,641 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,641 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,640 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,644 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,644 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,645 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,645 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,645 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,645 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,648 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/.splits/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31 for child: f7b3b5a8e1cf790554c2a165be1b8efb, parent: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,650 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31 for region: 3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:51,650 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 3208eb5903dfcc5b74b941b6cef73b91 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-11 03:55:51,680 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:51,682 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:51,683 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676087751683"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676087751683"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676087751683"}]},"ts":"1676087751683"} 2023-02-11 03:55:51,683 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087751683"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087751683"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087751683"}]},"ts":"1676087751683"} 2023-02-11 03:55:51,683 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087751683"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087751683"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087751683"}]},"ts":"1676087751683"} 2023-02-11 03:55:51,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, ASSIGN}] 2023-02-11 03:55:51,687 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, ASSIGN 2023-02-11 03:55:51,687 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:51,688 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, ASSIGN 2023-02-11 03:55:51,688 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36661,1676087644839; forceNewPlan=false, retain=false 2023-02-11 03:55:51,838 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:55:51,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=03f77114fde2a46ee94d8cbde75ff2bd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087751839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087751839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087751839"}]},"ts":"1676087751839"} 2023-02-11 03:55:51,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 03f77114fde2a46ee94d8cbde75ff2bd, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:51,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=f7b3b5a8e1cf790554c2a165be1b8efb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:51,843 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087751842"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087751842"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087751842"}]},"ts":"1676087751842"} 2023-02-11 03:55:51,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure f7b3b5a8e1cf790554c2a165be1b8efb, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:51,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:51,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f7b3b5a8e1cf790554c2a165be1b8efb, NAME => 'testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.', STARTKEY => '', ENDKEY => '71'} 2023-02-11 03:55:51,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:51,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:51,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:51,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:51,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:51,996 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:51,997 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1 2023-02-11 03:55:51,997 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1 2023-02-11 03:55:51,997 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f7b3b5a8e1cf790554c2a165be1b8efb columnFamilyName cf1 2023-02-11 03:55:52,002 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31 2023-02-11 03:55:52,004 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda 2023-02-11 03:55:52,007 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab 2023-02-11 03:55:52,009 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f 2023-02-11 03:55:52,012 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb 2023-02-11 03:55:52,014 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb 2023-02-11 03:55:52,016 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf 2023-02-11 03:55:52,016 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(310): Store=f7b3b5a8e1cf790554c2a165be1b8efb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:52,016 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,017 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2 2023-02-11 03:55:52,017 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2 2023-02-11 03:55:52,017 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f7b3b5a8e1cf790554c2a165be1b8efb columnFamilyName cf2 2023-02-11 03:55:52,022 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3 2023-02-11 03:55:52,024 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd 2023-02-11 03:55:52,027 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d 2023-02-11 03:55:52,029 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790 2023-02-11 03:55:52,031 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b 2023-02-11 03:55:52,034 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5 2023-02-11 03:55:52,036 DEBUG [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75 2023-02-11 03:55:52,036 INFO [StoreOpener-f7b3b5a8e1cf790554c2a165be1b8efb-1] regionserver.HStore(310): Store=f7b3b5a8e1cf790554c2a165be1b8efb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:52,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:52,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f7b3b5a8e1cf790554c2a165be1b8efb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68492802, jitterRate=0.020622283220291138}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:52,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f7b3b5a8e1cf790554c2a165be1b8efb: Running coprocessor pre-open hook at 1676087751995Writing region info on filesystem at 1676087751995Initializing all the Stores at 1676087751996 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087751996Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087751996Cleaning up temporary data from old regions at 1676087752038 (+42 ms)Cleaning up detritus from prior splits at 1676087752038Running coprocessor post-open hooks at 1676087752040 (+2 ms)Region opened successfully at 1676087752040 2023-02-11 03:55:52,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb., pid=134, masterSystemTime=1676087751992 2023-02-11 03:55:52,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:52,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1912): f7b3b5a8e1cf790554c2a165be1b8efb/cf1 is initiating minor compaction (all files) 2023-02-11 03:55:52,042 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1912): f7b3b5a8e1cf790554c2a165be1b8efb/cf2 is initiating minor compaction (all files) 2023-02-11 03:55:52,042 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f7b3b5a8e1cf790554c2a165be1b8efb/cf1 in testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,042 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f7b3b5a8e1cf790554c2a165be1b8efb/cf2 in testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,042 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/.tmp, totalSize=34.9 K 2023-02-11 03:55:52,042 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/.tmp, totalSize=34.9 K 2023-02-11 03:55:52,043 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087748761 2023-02-11 03:55:52,043 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087748761 2023-02-11 03:55:52,043 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087749328 2023-02-11 03:55:52,043 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087749328 2023-02-11 03:55:52,044 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087749897 2023-02-11 03:55:52,044 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=f7b3b5a8e1cf790554c2a165be1b8efb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:52,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,044 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087749897 2023-02-11 03:55:52,044 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752044"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087752044"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087752044"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087752044"}]},"ts":"1676087752044"} 2023-02-11 03:55:52,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,044 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087750067 2023-02-11 03:55:52,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 03f77114fde2a46ee94d8cbde75ff2bd, NAME => 'testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.', STARTKEY => '71', ENDKEY => ''} 2023-02-11 03:55:52,044 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087750067 2023-02-11 03:55:52,044 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087750639 2023-02-11 03:55:52,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:52,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:52,045 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087750639 2023-02-11 03:55:52,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,045 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087750803 2023-02-11 03:55:52,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,045 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087750803 2023-02-11 03:55:52,045 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087751370 2023-02-11 03:55:52,046 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087751370 2023-02-11 03:55:52,047 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-11 03:55:52,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure f7b3b5a8e1cf790554c2a165be1b8efb, server=jenkins-hbase4.apache.org,36661,1676087644839 in 202 msec 2023-02-11 03:55:52,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, ASSIGN in 362 msec 2023-02-11 03:55:52,049 DEBUG [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf1 2023-02-11 03:55:52,049 DEBUG [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf1 2023-02-11 03:55:52,050 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03f77114fde2a46ee94d8cbde75ff2bd columnFamilyName cf1 2023-02-11 03:55:52,050 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] regionserver.HStore(310): Store=03f77114fde2a46ee94d8cbde75ff2bd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:52,050 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,052 DEBUG [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf2 2023-02-11 03:55:52,052 DEBUG [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf2 2023-02-11 03:55:52,052 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03f77114fde2a46ee94d8cbde75ff2bd columnFamilyName cf2 2023-02-11 03:55:52,053 INFO [StoreOpener-03f77114fde2a46ee94d8cbde75ff2bd-1] regionserver.HStore(310): Store=03f77114fde2a46ee94d8cbde75ff2bd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:52,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:52,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 03f77114fde2a46ee94d8cbde75ff2bd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74078207, jitterRate=0.1038513034582138}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:52,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 03f77114fde2a46ee94d8cbde75ff2bd: Running coprocessor pre-open hook at 1676087752045Writing region info on filesystem at 1676087752045Initializing all the Stores at 1676087752046 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087752046Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087752047 (+1 ms)Cleaning up temporary data from old regions at 1676087752055 (+8 ms)Cleaning up detritus from prior splits at 1676087752056 (+1 ms)Running coprocessor post-open hooks at 1676087752059 (+3 ms)Region opened successfully at 1676087752059 2023-02-11 03:55:52,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd., pid=133, masterSystemTime=1676087751992 2023-02-11 03:55:52,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=03f77114fde2a46ee94d8cbde75ff2bd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:52,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752061"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087752061"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087752061"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087752061"}]},"ts":"1676087752061"} 2023-02-11 03:55:52,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-11 03:55:52,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 03f77114fde2a46ee94d8cbde75ff2bd, server=jenkins-hbase4.apache.org,36661,1676087644839 in 222 msec 2023-02-11 03:55:52,066 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] throttle.PressureAwareThroughputController(145): f7b3b5a8e1cf790554c2a165be1b8efb#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:52,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-11 03:55:52,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, ASSIGN in 378 msec 2023-02-11 03:55:52,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=3208eb5903dfcc5b74b941b6cef73b91, daughterA=f7b3b5a8e1cf790554c2a165be1b8efb, daughterB=03f77114fde2a46ee94d8cbde75ff2bd in 689 msec 2023-02-11 03:55:52,077 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] throttle.PressureAwareThroughputController(145): f7b3b5a8e1cf790554c2a165be1b8efb#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:52,090 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/.tmp/cf1/698b38fd20034df7a1be5e067c4909c5 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/698b38fd20034df7a1be5e067c4909c5 2023-02-11 03:55:52,092 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/.tmp/cf2/eae2d6d44386427cbbaa420704b64706 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/eae2d6d44386427cbbaa420704b64706 2023-02-11 03:55:52,094 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f7b3b5a8e1cf790554c2a165be1b8efb/cf1 of f7b3b5a8e1cf790554c2a165be1b8efb into 698b38fd20034df7a1be5e067c4909c5(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:52,094 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f7b3b5a8e1cf790554c2a165be1b8efb: 2023-02-11 03:55:52,094 INFO [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb., storeName=f7b3b5a8e1cf790554c2a165be1b8efb/cf1, priority=9, startTime=1676087752041; duration=0sec 2023-02-11 03:55:52,094 DEBUG [RS:0;jenkins-hbase4:36661-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:52,096 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f7b3b5a8e1cf790554c2a165be1b8efb/cf2 of f7b3b5a8e1cf790554c2a165be1b8efb into eae2d6d44386427cbbaa420704b64706(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:52,096 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f7b3b5a8e1cf790554c2a165be1b8efb: 2023-02-11 03:55:52,096 INFO [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb., storeName=f7b3b5a8e1cf790554c2a165be1b8efb/cf2, priority=9, startTime=1676087752042; duration=0sec 2023-02-11 03:55:52,096 DEBUG [RS:0;jenkins-hbase4:36661-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:52,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36661] ipc.CallRunner(144): callId: 989 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:49620 deadline: 1676087812130, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91. is not online on jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:52,239 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-11 03:55:52,242 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:52,243 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 03:55:52,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087752246"}]},"ts":"1676087752246"} 2023-02-11 03:55:52,247 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-11 03:55:52,249 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-11 03:55:52,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, UNASSIGN}] 2023-02-11 03:55:52,251 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, UNASSIGN 2023-02-11 03:55:52,251 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=03f77114fde2a46ee94d8cbde75ff2bd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:52,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752251"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087752251"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087752251"}]},"ts":"1676087752251"} 2023-02-11 03:55:52,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 03f77114fde2a46ee94d8cbde75ff2bd, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:52,257 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, UNASSIGN 2023-02-11 03:55:52,257 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=f7b3b5a8e1cf790554c2a165be1b8efb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:52,258 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752257"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087752257"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087752257"}]},"ts":"1676087752257"} 2023-02-11 03:55:52,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure f7b3b5a8e1cf790554c2a165be1b8efb, server=jenkins-hbase4.apache.org,36661,1676087644839}] 2023-02-11 03:55:52,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 03:55:52,404 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 03f77114fde2a46ee94d8cbde75ff2bd, disabling compactions & flushes 2023-02-11 03:55:52,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. after waiting 0 ms 2023-02-11 03:55:52,405 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,408 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-11 03:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:52,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd. 2023-02-11 03:55:52,409 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 03f77114fde2a46ee94d8cbde75ff2bd: Waiting for close lock at 1676087752404Running coprocessor pre-close hooks at 1676087752404Disabling compacts and flushes for region at 1676087752404Disabling writes for close at 1676087752405 (+1 ms)Writing region close event to WAL at 1676087752405Running coprocessor post-close hooks at 1676087752408 (+3 ms)Closed at 1676087752409 (+1 ms) 2023-02-11 03:55:52,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f7b3b5a8e1cf790554c2a165be1b8efb, disabling compactions & flushes 2023-02-11 03:55:52,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. after waiting 0 ms 2023-02-11 03:55:52,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,410 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=03f77114fde2a46ee94d8cbde75ff2bd, regionState=CLOSED 2023-02-11 03:55:52,410 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752410"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087752410"}]},"ts":"1676087752410"} 2023-02-11 03:55:52,411 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f] to archive 2023-02-11 03:55:52,411 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:52,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-11 03:55:52,413 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-4334acaa62b64baf8700a969ca0f48ab 2023-02-11 03:55:52,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 03f77114fde2a46ee94d8cbde75ff2bd, server=jenkins-hbase4.apache.org,36661,1676087644839 in 159 msec 2023-02-11 03:55:52,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=03f77114fde2a46ee94d8cbde75ff2bd, UNASSIGN in 163 msec 2023-02-11 03:55:52,414 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-2729682428bf4677be16f525bb80cfda 2023-02-11 03:55:52,416 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-26cb062853454228a0d5e82a03043d31 2023-02-11 03:55:52,417 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b1ed6b45f3b6472cbaf8ef8932974deb 2023-02-11 03:55:52,418 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-e5edea2a509b4612bab7941a039932bb 2023-02-11 03:55:52,419 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-fcbd60dd42224e88a36a71236af23ccf 2023-02-11 03:55:52,420 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-944593d4aff34b93a54031b2e289eb7f 2023-02-11 03:55:52,421 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd] to archive 2023-02-11 03:55:52,421 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:52,422 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-7d4d8ad9997442cc92e82863bb0e2790 2023-02-11 03:55:52,423 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b26bf95effe94bc8a536ee254d6888c5 2023-02-11 03:55:52,424 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-a3d54398dbe94b2da1e88bd2025b225b 2023-02-11 03:55:52,425 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-5a89582b37424402b9ab8e151843180d 2023-02-11 03:55:52,426 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-1b1d96fa8dc7472e8bc0a282078a0fb3 2023-02-11 03:55:52,427 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-b41e480c394f459691ff2a625d3a7f75 2023-02-11 03:55:52,427 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/testSplitTableRegionEmptyDaughter=3208eb5903dfcc5b74b941b6cef73b91-304ae36ff7644b3bb4df112db76016bd 2023-02-11 03:55:52,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:55:52,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:52,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb. 2023-02-11 03:55:52,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f7b3b5a8e1cf790554c2a165be1b8efb: Waiting for close lock at 1676087752410Running coprocessor pre-close hooks at 1676087752410Disabling compacts and flushes for region at 1676087752410Disabling writes for close at 1676087752410Writing region close event to WAL at 1676087752428 (+18 ms)Running coprocessor post-close hooks at 1676087752431 (+3 ms)Closed at 1676087752431 2023-02-11 03:55:52,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=f7b3b5a8e1cf790554c2a165be1b8efb, regionState=CLOSED 2023-02-11 03:55:52,432 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087752432"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087752432"}]},"ts":"1676087752432"} 2023-02-11 03:55:52,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-11 03:55:52,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure f7b3b5a8e1cf790554c2a165be1b8efb, server=jenkins-hbase4.apache.org,36661,1676087644839 in 175 msec 2023-02-11 03:55:52,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-02-11 03:55:52,435 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=f7b3b5a8e1cf790554c2a165be1b8efb, UNASSIGN in 184 msec 2023-02-11 03:55:52,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087752435"}]},"ts":"1676087752435"} 2023-02-11 03:55:52,436 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-11 03:55:52,438 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-11 03:55:52,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-02-11 03:55:52,526 ERROR [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 5cdfedec-baf9-457a-bd3c-897c5ed5a081) service to localhost/127.0.0.1:39467 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:55:52,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-11 03:55:52,548 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-11 03:55:52,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,551 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,551 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 03:55:52,555 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:52,555 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,555 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,557 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/recovered.edits] 2023-02-11 03:55:52,557 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/recovered.edits] 2023-02-11 03:55:52,557 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/recovered.edits] 2023-02-11 03:55:52,560 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/26cb062853454228a0d5e82a03043d31 2023-02-11 03:55:52,560 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/698b38fd20034df7a1be5e067c4909c5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf1/698b38fd20034df7a1be5e067c4909c5 2023-02-11 03:55:52,561 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/2729682428bf4677be16f525bb80cfda 2023-02-11 03:55:52,562 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/recovered.edits/88.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd/recovered.edits/88.seqid 2023-02-11 03:55:52,562 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/4334acaa62b64baf8700a969ca0f48ab 2023-02-11 03:55:52,562 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/eae2d6d44386427cbbaa420704b64706 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/cf2/eae2d6d44386427cbbaa420704b64706 2023-02-11 03:55:52,562 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/03f77114fde2a46ee94d8cbde75ff2bd 2023-02-11 03:55:52,563 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/944593d4aff34b93a54031b2e289eb7f 2023-02-11 03:55:52,564 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/b1ed6b45f3b6472cbaf8ef8932974deb 2023-02-11 03:55:52,565 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb/recovered.edits/90.seqid 2023-02-11 03:55:52,565 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/f7b3b5a8e1cf790554c2a165be1b8efb 2023-02-11 03:55:52,565 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/e5edea2a509b4612bab7941a039932bb 2023-02-11 03:55:52,566 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf1/fcbd60dd42224e88a36a71236af23ccf 2023-02-11 03:55:52,568 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/1b1d96fa8dc7472e8bc0a282078a0fb3 2023-02-11 03:55:52,569 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/304ae36ff7644b3bb4df112db76016bd 2023-02-11 03:55:52,570 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/5a89582b37424402b9ab8e151843180d 2023-02-11 03:55:52,571 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/7d4d8ad9997442cc92e82863bb0e2790 2023-02-11 03:55:52,572 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/a3d54398dbe94b2da1e88bd2025b225b 2023-02-11 03:55:52,573 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b26bf95effe94bc8a536ee254d6888c5 2023-02-11 03:55:52,573 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/cf2/b41e480c394f459691ff2a625d3a7f75 2023-02-11 03:55:52,575 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91/recovered.edits/85.seqid 2023-02-11 03:55:52,576 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegionEmptyDaughter/3208eb5903dfcc5b74b941b6cef73b91 2023-02-11 03:55:52,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-11 03:55:52,577 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,579 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-11 03:55:52,581 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-11 03:55:52,582 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,582 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-11 03:55:52,582 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087752582"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:52,582 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087752582"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:52,582 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087752582"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:52,584 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:55:52,584 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3208eb5903dfcc5b74b941b6cef73b91, NAME => 'testSplitTableRegionEmptyDaughter,,1676087747757.3208eb5903dfcc5b74b941b6cef73b91.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f7b3b5a8e1cf790554c2a165be1b8efb, NAME => 'testSplitTableRegionEmptyDaughter,,1676087751378.f7b3b5a8e1cf790554c2a165be1b8efb.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 03f77114fde2a46ee94d8cbde75ff2bd, NAME => 'testSplitTableRegionEmptyDaughter,71,1676087751378.03f77114fde2a46ee94d8cbde75ff2bd.', STARTKEY => '71', ENDKEY => ''}] 2023-02-11 03:55:52,584 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-11 03:55:52,584 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087752584"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:52,585 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-11 03:55:52,587 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-11 03:55:52,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 39 msec 2023-02-11 03:55:52,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-11 03:55:52,653 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-11 03:55:52,666 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=371 (was 374), OpenFileDescriptor=838 (was 842), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=118 (was 94) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2949 (was 2941) - AvailableMemoryMB LEAK? - 2023-02-11 03:55:52,681 INFO [Listener at localhost/45015] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=371, OpenFileDescriptor=838, MaxFileDescriptor=60000, SystemLoadAverage=118, ProcessCount=168, AvailableMemoryMB=2949 2023-02-11 03:55:52,681 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:52,685 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-11 03:55:52,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-11 03:55:52,687 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-11 03:55:52,688 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:52,689 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:52,690 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:52,691 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 empty. 2023-02-11 03:55:52,691 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:52,691 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 03:55:52,703 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:52,704 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 941c0ecb516230bdc53f83554817cc22, NAME => 'testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 941c0ecb516230bdc53f83554817cc22, disabling compactions & flushes 2023-02-11 03:55:52,711 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. after waiting 0 ms 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:52,711 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:52,711 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 941c0ecb516230bdc53f83554817cc22: Waiting for close lock at 1676087752711Disabling compacts and flushes for region at 1676087752711Disabling writes for close at 1676087752711Writing region close event to WAL at 1676087752711Closed at 1676087752711 2023-02-11 03:55:52,713 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:52,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087752714"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087752714"}]},"ts":"1676087752714"} 2023-02-11 03:55:52,715 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:52,715 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:52,715 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087752715"}]},"ts":"1676087752715"} 2023-02-11 03:55:52,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-11 03:55:52,720 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-11 03:55:52,721 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-11 03:55:52,721 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-11 03:55:52,721 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-11 03:55:52,721 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-11 03:55:52,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, ASSIGN}] 2023-02-11 03:55:52,722 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, ASSIGN 2023-02-11 03:55:52,723 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:55:52,873 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-11 03:55:52,874 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=941c0ecb516230bdc53f83554817cc22, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:52,874 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087752874"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087752874"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087752874"}]},"ts":"1676087752874"} 2023-02-11 03:55:52,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 941c0ecb516230bdc53f83554817cc22, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:53,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 941c0ecb516230bdc53f83554817cc22, NAME => 'testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:53,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:53,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:53,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,034 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,035 DEBUG [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1 2023-02-11 03:55:53,035 DEBUG [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1 2023-02-11 03:55:53,035 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 941c0ecb516230bdc53f83554817cc22 columnFamilyName cf1 2023-02-11 03:55:53,035 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] regionserver.HStore(310): Store=941c0ecb516230bdc53f83554817cc22/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:53,035 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,036 DEBUG [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2 2023-02-11 03:55:53,036 DEBUG [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2 2023-02-11 03:55:53,037 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 941c0ecb516230bdc53f83554817cc22 columnFamilyName cf2 2023-02-11 03:55:53,037 INFO [StoreOpener-941c0ecb516230bdc53f83554817cc22-1] regionserver.HStore(310): Store=941c0ecb516230bdc53f83554817cc22/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:53,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:53,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:53,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:53,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 941c0ecb516230bdc53f83554817cc22; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74879958, jitterRate=0.11579832434654236}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:53,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 941c0ecb516230bdc53f83554817cc22: Running coprocessor pre-open hook at 1676087753033Writing region info on filesystem at 1676087753033Initializing all the Stores at 1676087753033Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087753033Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087753033Cleaning up temporary data from old regions at 1676087753038 (+5 ms)Cleaning up detritus from prior splits at 1676087753039 (+1 ms)Running coprocessor post-open hooks at 1676087753042 (+3 ms)Region opened successfully at 1676087753042 2023-02-11 03:55:53,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22., pid=143, masterSystemTime=1676087753027 2023-02-11 03:55:53,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,044 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=941c0ecb516230bdc53f83554817cc22, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,044 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087753044"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087753044"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087753044"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087753044"}]},"ts":"1676087753044"} 2023-02-11 03:55:53,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-11 03:55:53,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 941c0ecb516230bdc53f83554817cc22, server=jenkins-hbase4.apache.org,33925,1676087644887 in 169 msec 2023-02-11 03:55:53,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-11 03:55:53,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, ASSIGN in 325 msec 2023-02-11 03:55:53,048 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:53,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087753048"}]},"ts":"1676087753048"} 2023-02-11 03:55:53,049 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-11 03:55:53,050 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:53,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 364 msec 2023-02-11 03:55:53,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:53,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:53,195 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:53,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:53,195 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:53,196 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,196 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,197 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,197 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,197 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,197 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:53,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,198 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,198 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:53,199 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:53,199 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,199 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:53,200 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-11 03:55:53,201 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,201 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,201 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,201 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,201 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,202 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,213 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/7f8149e35be54518a65375dd2b693659 2023-02-11 03:55:53,223 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/1ba5936892f3464281b78f79f8465110 2023-02-11 03:55:53,226 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/7f8149e35be54518a65375dd2b693659 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659 2023-02-11 03:55:53,229 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:53,229 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/1ba5936892f3464281b78f79f8465110 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110 2023-02-11 03:55:53,232 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:53,232 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 32ms, sequenceid=6, compaction requested=false 2023-02-11 03:55:53,232 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-11 03:55:53,233 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:53,233 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,233 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:53,233 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:53,233 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,233 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,233 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,234 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,234 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,235 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,235 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:53,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39a8563b[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:53,236 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:53,236 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,237 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:53,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,238 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,238 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:53,238 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,239 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,244 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,244 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,244 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,244 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:53,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:53,247 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:53,247 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:53,247 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:53,247 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:53,247 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:53,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,249 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,249 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,250 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,251 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,251 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,252 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:53,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:53,254 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:53,254 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,255 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,355 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:53,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:53,365 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:53,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:53,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:53,366 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:53,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:53,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:53,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,366 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,367 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,367 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,367 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,367 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,368 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,368 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,368 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,369 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,369 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:53,370 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,370 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:53,370 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:53,371 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,372 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,373 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,382 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/ed43a5800e0442478419adcc729cb4ec 2023-02-11 03:55:53,393 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/ee9818bb91284cb1b503a67eaac2764e 2023-02-11 03:55:53,396 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/ed43a5800e0442478419adcc729cb4ec as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec 2023-02-11 03:55:53,399 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:53,400 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/ee9818bb91284cb1b503a67eaac2764e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e 2023-02-11 03:55:53,403 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-11 03:55:53,404 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 34ms, sequenceid=19, compaction requested=false 2023-02-11 03:55:53,404 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:53,404 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:53,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:53,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,405 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,406 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,406 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,407 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,407 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:53,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@373b66ff[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:53,408 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:53,408 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,409 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,409 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,409 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,410 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,410 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:53,410 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,413 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:53,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:53,416 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:53,416 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:53,416 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:53,416 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:53,416 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,418 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,418 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,419 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,419 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,421 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,422 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,422 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,425 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:53,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:53,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,425 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:53,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,426 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,526 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:53,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:53,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:53,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:53,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:53,538 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:53,538 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,538 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:53,538 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:53,539 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,539 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,540 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,540 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,540 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:53,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,541 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,542 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,542 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:53,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,542 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:53,542 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,542 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,542 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,546 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,552 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 2023-02-11 03:55:53,572 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 2023-02-11 03:55:53,576 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 2023-02-11 03:55:53,579 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:53,580 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 2023-02-11 03:55:53,583 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-11 03:55:53,584 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 42ms, sequenceid=32, compaction requested=true 2023-02-11 03:55:53,584 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:53,584 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,584 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:53,584 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:53,584 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,584 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,584 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,586 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,586 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,587 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:53,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22afd456[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:53,588 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:53,588 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,589 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,589 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,590 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:53,591 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,592 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,591 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,592 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,592 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:53,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:53,594 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:53,594 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:53,594 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:53,594 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:53,594 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,596 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,597 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,597 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,597 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,598 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,599 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,600 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:53,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:53,602 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,602 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,603 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,703 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:53,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:53,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:53,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:53,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:53,715 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:53,715 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,715 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:53,715 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:53,716 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,716 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,717 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,717 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,717 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,717 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,718 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,718 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,718 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,718 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:53,719 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,719 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,719 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:53,719 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:53,720 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,724 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,724 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,724 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,725 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,725 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,729 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/3acd20ba935a4459b53a1674ffa3f0a1 2023-02-11 03:55:53,739 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/e8878420edff49b2a3137bfd51785d05 2023-02-11 03:55:53,742 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/3acd20ba935a4459b53a1674ffa3f0a1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1 2023-02-11 03:55:53,745 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:53,746 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/e8878420edff49b2a3137bfd51785d05 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05 2023-02-11 03:55:53,749 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05, entries=10, sequenceid=45, filesize=5.0 K 2023-02-11 03:55:53,749 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 30ms, sequenceid=45, compaction requested=true 2023-02-11 03:55:53,750 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:53,750 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,750 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,751 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,751 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,751 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,751 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:53,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b91b639[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:53,753 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:53,753 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:53,754 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,754 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:53,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,757 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,758 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:53,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:53,761 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:53,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:53,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:53,761 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:53,761 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,762 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,763 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,763 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,764 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,765 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,766 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,766 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:53,772 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:53,772 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:53,772 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:53,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,773 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,873 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:53,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:53,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:53,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:53,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:53,885 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:53,885 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,885 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:53,885 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:53,886 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,886 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,888 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,888 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,888 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,888 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,889 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,889 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:53,889 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:53,890 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:53,890 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,890 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:53,890 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:53,891 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,891 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,891 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,891 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,891 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,892 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,904 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/1390d400dda84c96b763e4650c273b98 2023-02-11 03:55:53,914 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/847047970a7342ca9f8e0ddd18892727 2023-02-11 03:55:53,917 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/1390d400dda84c96b763e4650c273b98 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98 2023-02-11 03:55:53,920 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:53,920 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/847047970a7342ca9f8e0ddd18892727 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727 2023-02-11 03:55:53,923 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727, entries=10, sequenceid=58, filesize=5.0 K 2023-02-11 03:55:53,924 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 34ms, sequenceid=58, compaction requested=true 2023-02-11 03:55:53,924 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:53,924 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:53,924 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:53,926 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,926 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:53,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@307328fe[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:53,928 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:53,928 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,929 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,929 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:53,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:53,929 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,935 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,936 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,936 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:53,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:53,939 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:53,939 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:53,939 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:53,939 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:53,939 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,940 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,940 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:53,941 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,941 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,943 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:53,943 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:53,943 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:53,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,946 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:53,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:53,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,946 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:53,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:53,947 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,047 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:54,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:54,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:54,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:54,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-11 03:55:54,059 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-11 03:55:54,060 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:54,060 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-11 03:55:54,060 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-11 03:55:54,060 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,060 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,062 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,062 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,062 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:54,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-11 03:55:54,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:54,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:54,063 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36661,1676087644839' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:54,064 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:54,064 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-11 03:55:54,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:54,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36767,1676087644919' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:55:54,064 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:54,064 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,064 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. started... 2023-02-11 03:55:54,064 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-11 03:55:54,065 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,066 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:54,073 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 2023-02-11 03:55:54,082 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/f0615b5d076841bca4178af84f38ad9a 2023-02-11 03:55:54,085 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 2023-02-11 03:55:54,088 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:54,089 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/f0615b5d076841bca4178af84f38ad9a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a 2023-02-11 03:55:54,092 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-11 03:55:54,093 INFO [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 29ms, sequenceid=71, compaction requested=true 2023-02-11 03:55:54,093 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 941c0ecb516230bdc53f83554817cc22: 2023-02-11 03:55:54,093 DEBUG [rs(jenkins-hbase4.apache.org,33925,1676087644887)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-11 03:55:54,093 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-11 03:55:54,094 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,094 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:54,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,095 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:54,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33925,1676087644887' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-11 03:55:54,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c93ac91[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:54,096 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-11 03:55:54,096 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:54,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,102 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:54,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,101 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36767,1676087644919' in zk 2023-02-11 03:55:54,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33925,1676087644887' in zk 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-11 03:55:54,102 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36661,1676087644839' in zk 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36767,1676087644919' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:54,111 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-11 03:55:54,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,33925,1676087644887' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:54,111 DEBUG [member: 'jenkins-hbase4.apache.org,36661,1676087644839' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-11 03:55:54,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:54,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:54,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:54,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:54,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:54,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33925,1676087644887': 2023-02-11 03:55:54,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33925,1676087644887' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-11 03:55:54,114 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-11 03:55:54,114 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:54,114 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:54,114 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-11 03:55:54,114 INFO [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:54,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:54,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:54,118 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,117 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:54,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:54,118 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,118 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:54,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,120 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:54,120 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-11 03:55:54,120 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,124 DEBUG [(jenkins-hbase4.apache.org,36397,1676087643676)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-11 03:55:54,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:54,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:54,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-11 03:55:54,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:54,124 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:54,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:54,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:54,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-11 03:55:54,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:54,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:54,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-11 03:55:54,125 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-11 03:55:54,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:54,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:54,225 DEBUG [Listener at localhost/45015] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:54,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:54,232 DEBUG [Listener at localhost/45015] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,233 DEBUG [Listener at localhost/45015] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=941c0ecb516230bdc53f83554817cc22, daughterA=e29181dea4ebd71fa849a2f526b3b451, daughterB=74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:54,234 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=941c0ecb516230bdc53f83554817cc22, daughterA=e29181dea4ebd71fa849a2f526b3b451, daughterB=74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:54,234 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=941c0ecb516230bdc53f83554817cc22, daughterA=e29181dea4ebd71fa849a2f526b3b451, daughterB=74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:54,234 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=941c0ecb516230bdc53f83554817cc22, daughterA=e29181dea4ebd71fa849a2f526b3b451, daughterB=74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:54,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, UNASSIGN}] 2023-02-11 03:55:54,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, UNASSIGN 2023-02-11 03:55:54,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=941c0ecb516230bdc53f83554817cc22, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:54,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087754236"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087754236"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087754236"}]},"ts":"1676087754236"} 2023-02-11 03:55:54,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 941c0ecb516230bdc53f83554817cc22, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:54,388 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 941c0ecb516230bdc53f83554817cc22, disabling compactions & flushes 2023-02-11 03:55:54,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. after waiting 0 ms 2023-02-11 03:55:54,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 941c0ecb516230bdc53f83554817cc22 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-11 03:55:54,397 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/1afdb9bc3986411ba39c4cff6def5c0f 2023-02-11 03:55:54,407 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/ae4593f8ead64db79efd0ed0d244e59e 2023-02-11 03:55:54,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf1/1afdb9bc3986411ba39c4cff6def5c0f as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f 2023-02-11 03:55:54,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:54,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/.tmp/cf2/ae4593f8ead64db79efd0ed0d244e59e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e 2023-02-11 03:55:54,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-11 03:55:54,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 27ms, sequenceid=82, compaction requested=true 2023-02-11 03:55:54,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-11 03:55:54,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:54,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. 2023-02-11 03:55:54,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 941c0ecb516230bdc53f83554817cc22: Waiting for close lock at 1676087754389Running coprocessor pre-close hooks at 1676087754389Disabling compacts and flushes for region at 1676087754389Disabling writes for close at 1676087754389Obtaining lock to block concurrent updates at 1676087754389Preparing flush snapshotting stores in 941c0ecb516230bdc53f83554817cc22 at 1676087754389Finished memstore snapshotting testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676087754389Flushing stores of testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. at 1676087754389Flushing 941c0ecb516230bdc53f83554817cc22/cf1: creating writer at 1676087754390 (+1 ms)Flushing 941c0ecb516230bdc53f83554817cc22/cf1: appending metadata at 1676087754392 (+2 ms)Flushing 941c0ecb516230bdc53f83554817cc22/cf1: closing flushed file at 1676087754392Flushing 941c0ecb516230bdc53f83554817cc22/cf2: creating writer at 1676087754400 (+8 ms)Flushing 941c0ecb516230bdc53f83554817cc22/cf2: appending metadata at 1676087754402 (+2 ms)Flushing 941c0ecb516230bdc53f83554817cc22/cf2: closing flushed file at 1676087754402Flushing 941c0ecb516230bdc53f83554817cc22/cf1: reopening flushed file at 1676087754410 (+8 ms)Flushing 941c0ecb516230bdc53f83554817cc22/cf2: reopening flushed file at 1676087754413 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 941c0ecb516230bdc53f83554817cc22 in 27ms, sequenceid=82, compaction requested=true at 1676087754417 (+4 ms)Writing region close event to WAL at 1676087754418 (+1 ms)Running coprocessor post-close hooks at 1676087754421 (+3 ms)Closed at 1676087754421 2023-02-11 03:55:54,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=941c0ecb516230bdc53f83554817cc22, regionState=CLOSED 2023-02-11 03:55:54,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087754423"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087754423"}]},"ts":"1676087754423"} 2023-02-11 03:55:54,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-11 03:55:54,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 941c0ecb516230bdc53f83554817cc22, server=jenkins-hbase4.apache.org,33925,1676087644887 in 187 msec 2023-02-11 03:55:54,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-11 03:55:54,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=941c0ecb516230bdc53f83554817cc22, UNASSIGN in 190 msec 2023-02-11 03:55:54,435 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=941c0ecb516230bdc53f83554817cc22, threads=14 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,435 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,436 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,525 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,526 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,532 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,532 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,533 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,534 ERROR [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 328ac628-71ce-4e94-9ad2-68da08fa646b) service to localhost/127.0.0.1:39467 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:55:54,540 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,541 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,542 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,545 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,545 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,546 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,546 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:54,547 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:55,285 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05 for region: 941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:55,285 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 941c0ecb516230bdc53f83554817cc22 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-11 03:55:55,317 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:55,318 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-11 03:55:55,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676087755319"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676087755319"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676087755319"}]},"ts":"1676087755319"} 2023-02-11 03:55:55,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755319"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087755319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087755319"}]},"ts":"1676087755319"} 2023-02-11 03:55:55,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755319"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087755319"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087755319"}]},"ts":"1676087755319"} 2023-02-11 03:55:55,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, ASSIGN}] 2023-02-11 03:55:55,322 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, ASSIGN 2023-02-11 03:55:55,322 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:55:55,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, ASSIGN 2023-02-11 03:55:55,323 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33925,1676087644887; forceNewPlan=false, retain=false 2023-02-11 03:55:55,473 INFO [jenkins-hbase4:36397] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-11 03:55:55,474 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=74e1d76b732588b92d3ddfdecf8ade4c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:55,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755474"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087755474"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087755474"}]},"ts":"1676087755474"} 2023-02-11 03:55:55,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 74e1d76b732588b92d3ddfdecf8ade4c, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:55,477 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e29181dea4ebd71fa849a2f526b3b451, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:55,478 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755477"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087755477"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087755477"}]},"ts":"1676087755477"} 2023-02-11 03:55:55,478 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure e29181dea4ebd71fa849a2f526b3b451, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:55,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e29181dea4ebd71fa849a2f526b3b451, NAME => 'testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.', STARTKEY => '', ENDKEY => '41'} 2023-02-11 03:55:55,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,631 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,632 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1 2023-02-11 03:55:55,632 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1 2023-02-11 03:55:55,632 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e29181dea4ebd71fa849a2f526b3b451 columnFamilyName cf1 2023-02-11 03:55:55,644 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-bottom 2023-02-11 03:55:55,647 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-bottom 2023-02-11 03:55:55,651 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-bottom 2023-02-11 03:55:55,654 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-bottom 2023-02-11 03:55:55,658 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-bottom 2023-02-11 03:55:55,661 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-bottom 2023-02-11 03:55:55,665 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-bottom 2023-02-11 03:55:55,665 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(310): Store=e29181dea4ebd71fa849a2f526b3b451/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:55,665 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,666 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2 2023-02-11 03:55:55,666 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2 2023-02-11 03:55:55,666 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e29181dea4ebd71fa849a2f526b3b451 columnFamilyName cf2 2023-02-11 03:55:55,677 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-bottom 2023-02-11 03:55:55,681 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-bottom 2023-02-11 03:55:55,683 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-bottom 2023-02-11 03:55:55,687 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-bottom 2023-02-11 03:55:55,690 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-bottom 2023-02-11 03:55:55,693 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-bottom 2023-02-11 03:55:55,697 DEBUG [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-bottom 2023-02-11 03:55:55,697 INFO [StoreOpener-e29181dea4ebd71fa849a2f526b3b451-1] regionserver.HStore(310): Store=e29181dea4ebd71fa849a2f526b3b451/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:55,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:55,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:55,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e29181dea4ebd71fa849a2f526b3b451; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60747963, jitterRate=-0.09478481113910675}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:55,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e29181dea4ebd71fa849a2f526b3b451: Running coprocessor pre-open hook at 1676087755630Writing region info on filesystem at 1676087755630Initializing all the Stores at 1676087755631 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087755631Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087755631Cleaning up temporary data from old regions at 1676087755699 (+68 ms)Cleaning up detritus from prior splits at 1676087755699Running coprocessor post-open hooks at 1676087755701 (+2 ms)Region opened successfully at 1676087755701 2023-02-11 03:55:55,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451., pid=150, masterSystemTime=1676087755627 2023-02-11 03:55:55,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:55,702 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:55,702 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:55,702 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,702 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,702 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): e29181dea4ebd71fa849a2f526b3b451/cf1 is initiating minor compaction (all files) 2023-02-11 03:55:55,702 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): e29181dea4ebd71fa849a2f526b3b451/cf2 is initiating minor compaction (all files) 2023-02-11 03:55:55,703 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e29181dea4ebd71fa849a2f526b3b451/cf1 in testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,703 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e29181dea4ebd71fa849a2f526b3b451/cf2 in testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,703 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/.tmp, totalSize=34.9 K 2023-02-11 03:55:55,703 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-bottom] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/.tmp, totalSize=34.9 K 2023-02-11 03:55:55,703 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087753190 2023-02-11 03:55:55,703 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676087753190 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:55,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,704 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087753356 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 74e1d76b732588b92d3ddfdecf8ade4c, NAME => 'testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.', STARTKEY => '41', ENDKEY => ''} 2023-02-11 03:55:55,704 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676087753356 2023-02-11 03:55:55,704 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=e29181dea4ebd71fa849a2f526b3b451, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:55,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-11 03:55:55,704 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755704"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087755704"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087755704"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087755704"}]},"ts":"1676087755704"} 2023-02-11 03:55:55,704 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087753527 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,705 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676087753527 2023-02-11 03:55:55,705 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087753705 2023-02-11 03:55:55,705 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676087753705 2023-02-11 03:55:55,705 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087753875 2023-02-11 03:55:55,706 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676087753875 2023-02-11 03:55:55,706 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087754048 2023-02-11 03:55:55,706 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676087754048 2023-02-11 03:55:55,706 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087754226 2023-02-11 03:55:55,706 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,706 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676087754226 2023-02-11 03:55:55,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-02-11 03:55:55,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure e29181dea4ebd71fa849a2f526b3b451, server=jenkins-hbase4.apache.org,33925,1676087644887 in 227 msec 2023-02-11 03:55:55,708 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1 2023-02-11 03:55:55,708 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1 2023-02-11 03:55:55,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, ASSIGN in 387 msec 2023-02-11 03:55:55,709 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74e1d76b732588b92d3ddfdecf8ade4c columnFamilyName cf1 2023-02-11 03:55:55,743 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): e29181dea4ebd71fa849a2f526b3b451#cf2#compaction#142 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:55,743 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): e29181dea4ebd71fa849a2f526b3b451#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:55,748 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-top 2023-02-11 03:55:55,750 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-top 2023-02-11 03:55:55,753 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-top 2023-02-11 03:55:55,760 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-top 2023-02-11 03:55:55,766 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-top 2023-02-11 03:55:55,768 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/.tmp/cf2/bfdcc4331fb54a85ba0abf7a5d060d91 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/bfdcc4331fb54a85ba0abf7a5d060d91 2023-02-11 03:55:55,768 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/.tmp/cf1/0eb6ce937dd449fbb86df9a70c8b1ac1 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/0eb6ce937dd449fbb86df9a70c8b1ac1 2023-02-11 03:55:55,770 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-top 2023-02-11 03:55:55,772 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e29181dea4ebd71fa849a2f526b3b451/cf2 of e29181dea4ebd71fa849a2f526b3b451 into bfdcc4331fb54a85ba0abf7a5d060d91(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:55,772 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e29181dea4ebd71fa849a2f526b3b451: 2023-02-11 03:55:55,772 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451., storeName=e29181dea4ebd71fa849a2f526b3b451/cf2, priority=9, startTime=1676087755702; duration=0sec 2023-02-11 03:55:55,772 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e29181dea4ebd71fa849a2f526b3b451/cf1 of e29181dea4ebd71fa849a2f526b3b451 into 0eb6ce937dd449fbb86df9a70c8b1ac1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:55,772 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,772 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e29181dea4ebd71fa849a2f526b3b451: 2023-02-11 03:55:55,772 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451., storeName=e29181dea4ebd71fa849a2f526b3b451/cf1, priority=9, startTime=1676087755701; duration=0sec 2023-02-11 03:55:55,773 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,773 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-top 2023-02-11 03:55:55,773 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(310): Store=74e1d76b732588b92d3ddfdecf8ade4c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:55,773 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,774 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2 2023-02-11 03:55:55,774 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2 2023-02-11 03:55:55,774 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 74e1d76b732588b92d3ddfdecf8ade4c columnFamilyName cf2 2023-02-11 03:55:55,783 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-top 2023-02-11 03:55:55,786 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-top 2023-02-11 03:55:55,788 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-top 2023-02-11 03:55:55,791 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-top 2023-02-11 03:55:55,793 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-top 2023-02-11 03:55:55,796 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-top 2023-02-11 03:55:55,799 DEBUG [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(539): loaded hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-top 2023-02-11 03:55:55,799 INFO [StoreOpener-74e1d76b732588b92d3ddfdecf8ade4c-1] regionserver.HStore(310): Store=74e1d76b732588b92d3ddfdecf8ade4c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:55,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:55,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:55,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 74e1d76b732588b92d3ddfdecf8ade4c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72733669, jitterRate=0.08381612598896027}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:55,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 74e1d76b732588b92d3ddfdecf8ade4c: Running coprocessor pre-open hook at 1676087755705Writing region info on filesystem at 1676087755705Initializing all the Stores at 1676087755705Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087755706 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087755706Cleaning up temporary data from old regions at 1676087755800 (+94 ms)Cleaning up detritus from prior splits at 1676087755801 (+1 ms)Running coprocessor post-open hooks at 1676087755802 (+1 ms)Region opened successfully at 1676087755802 2023-02-11 03:55:55,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c., pid=149, masterSystemTime=1676087755627 2023-02-11 03:55:55,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:55,804 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:55,804 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-11 03:55:55,804 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,804 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1912): 74e1d76b732588b92d3ddfdecf8ade4c/cf1 is initiating minor compaction (all files) 2023-02-11 03:55:55,804 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,804 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 74e1d76b732588b92d3ddfdecf8ade4c/cf1 in testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,804 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1912): 74e1d76b732588b92d3ddfdecf8ade4c/cf2 is initiating minor compaction (all files) 2023-02-11 03:55:55,804 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 74e1d76b732588b92d3ddfdecf8ade4c/cf2 in testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,804 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/.tmp, totalSize=34.9 K 2023-02-11 03:55:55,805 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-top] into tmpdir=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/.tmp, totalSize=34.9 K 2023-02-11 03:55:55,805 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087753190 2023-02-11 03:55:55,805 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676087753190 2023-02-11 03:55:55,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,805 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087753356 2023-02-11 03:55:55,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:55,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=74e1d76b732588b92d3ddfdecf8ade4c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:55,806 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676087753356 2023-02-11 03:55:55,806 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087753527 2023-02-11 03:55:55,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087755805"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087755805"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087755805"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087755805"}]},"ts":"1676087755805"} 2023-02-11 03:55:55,806 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087753705 2023-02-11 03:55:55,806 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676087753527 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087753875 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676087753705 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087754048 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting 847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676087753875 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] compactions.Compactor(207): Compacting 1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087754226 2023-02-11 03:55:55,807 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676087754048 2023-02-11 03:55:55,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-02-11 03:55:55,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 74e1d76b732588b92d3ddfdecf8ade4c, server=jenkins-hbase4.apache.org,33925,1676087644887 in 331 msec 2023-02-11 03:55:55,808 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] compactions.Compactor(207): Compacting ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676087754226 2023-02-11 03:55:55,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-11 03:55:55,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, ASSIGN in 487 msec 2023-02-11 03:55:55,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=941c0ecb516230bdc53f83554817cc22, daughterA=e29181dea4ebd71fa849a2f526b3b451, daughterB=74e1d76b732588b92d3ddfdecf8ade4c in 1.5780 sec 2023-02-11 03:55:55,817 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] throttle.PressureAwareThroughputController(145): 74e1d76b732588b92d3ddfdecf8ade4c#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:55,817 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] throttle.PressureAwareThroughputController(145): 74e1d76b732588b92d3ddfdecf8ade4c#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:55,826 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/.tmp/cf1/9ec9706bcc34406e8ec478e4ffa304aa as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/9ec9706bcc34406e8ec478e4ffa304aa 2023-02-11 03:55:55,826 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/.tmp/cf2/14e946de06884295a346074b907fd411 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/14e946de06884295a346074b907fd411 2023-02-11 03:55:55,830 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 74e1d76b732588b92d3ddfdecf8ade4c/cf1 of 74e1d76b732588b92d3ddfdecf8ade4c into 9ec9706bcc34406e8ec478e4ffa304aa(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:55,830 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 74e1d76b732588b92d3ddfdecf8ade4c/cf2 of 74e1d76b732588b92d3ddfdecf8ade4c into 14e946de06884295a346074b907fd411(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:55,830 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 74e1d76b732588b92d3ddfdecf8ade4c: 2023-02-11 03:55:55,830 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 74e1d76b732588b92d3ddfdecf8ade4c: 2023-02-11 03:55:55,830 INFO [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c., storeName=74e1d76b732588b92d3ddfdecf8ade4c/cf1, priority=9, startTime=1676087755803; duration=0sec 2023-02-11 03:55:55,830 INFO [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c., storeName=74e1d76b732588b92d3ddfdecf8ade4c/cf2, priority=9, startTime=1676087755804; duration=0sec 2023-02-11 03:55:55,830 DEBUG [RS:1;jenkins-hbase4:33925-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,830 DEBUG [RS:1;jenkins-hbase4:33925-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:55,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33925] ipc.CallRunner(144): callId: 1118 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:55758 deadline: 1676087815994, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22. is not online on jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:56,103 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:55:56,109 INFO [Listener at localhost/45015] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-11 03:55:56,110 DEBUG [Listener at localhost/45015] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-11 03:55:56,115 WARN [Listener at localhost/45015] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-11 03:55:56,116 INFO [Listener at localhost/45015] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-11 03:55:56,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-11 03:55:56,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 03:55:56,119 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087756119"}]},"ts":"1676087756119"} 2023-02-11 03:55:56,120 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-11 03:55:56,122 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-11 03:55:56,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, UNASSIGN}] 2023-02-11 03:55:56,123 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, UNASSIGN 2023-02-11 03:55:56,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=74e1d76b732588b92d3ddfdecf8ade4c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:56,124 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087756124"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087756124"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087756124"}]},"ts":"1676087756124"} 2023-02-11 03:55:56,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 74e1d76b732588b92d3ddfdecf8ade4c, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:56,126 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, UNASSIGN 2023-02-11 03:55:56,127 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e29181dea4ebd71fa849a2f526b3b451, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:56,127 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087756127"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087756127"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087756127"}]},"ts":"1676087756127"} 2023-02-11 03:55:56,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure e29181dea4ebd71fa849a2f526b3b451, server=jenkins-hbase4.apache.org,33925,1676087644887}] 2023-02-11 03:55:56,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 03:55:56,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:56,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 74e1d76b732588b92d3ddfdecf8ade4c, disabling compactions & flushes 2023-02-11 03:55:56,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:56,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:56,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. after waiting 0 ms 2023-02-11 03:55:56,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:56,277 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-top] to archive 2023-02-11 03:55:56,278 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:56,279 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,280 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,281 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,282 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,283 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,284 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,285 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,286 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-top, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-top] to archive 2023-02-11 03:55:56,286 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:56,287 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,288 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,289 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,290 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,291 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,292 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,293 DEBUG [StoreCloser-testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:55:56,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:56,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c. 2023-02-11 03:55:56,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 74e1d76b732588b92d3ddfdecf8ade4c: Waiting for close lock at 1676087756276Running coprocessor pre-close hooks at 1676087756276Disabling compacts and flushes for region at 1676087756276Disabling writes for close at 1676087756276Writing region close event to WAL at 1676087756294 (+18 ms)Running coprocessor post-close hooks at 1676087756296 (+2 ms)Closed at 1676087756296 2023-02-11 03:55:56,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:56,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:56,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e29181dea4ebd71fa849a2f526b3b451, disabling compactions & flushes 2023-02-11 03:55:56,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:56,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:56,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. after waiting 0 ms 2023-02-11 03:55:56,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:56,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=74e1d76b732588b92d3ddfdecf8ade4c, regionState=CLOSED 2023-02-11 03:55:56,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087756298"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087756298"}]},"ts":"1676087756298"} 2023-02-11 03:55:56,299 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f-bottom] to archive 2023-02-11 03:55:56,299 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:56,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-11 03:55:56,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 74e1d76b732588b92d3ddfdecf8ade4c, server=jenkins-hbase4.apache.org,33925,1676087644887 in 174 msec 2023-02-11 03:55:56,301 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/7f8149e35be54518a65375dd2b693659.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=74e1d76b732588b92d3ddfdecf8ade4c, UNASSIGN in 177 msec 2023-02-11 03:55:56,302 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/ed43a5800e0442478419adcc729cb4ec.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,302 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,303 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/3acd20ba935a4459b53a1674ffa3f0a1.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,304 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1390d400dda84c96b763e4650c273b98.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,305 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/28ce3f4f82c34e4c9315c0a68865c6b2.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,306 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/1afdb9bc3986411ba39c4cff6def5c0f.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,307 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a-bottom, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22->hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e-bottom] to archive 2023-02-11 03:55:56,308 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:56,309 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/1ba5936892f3464281b78f79f8465110.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,310 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ee9818bb91284cb1b503a67eaac2764e.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,311 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,311 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/e8878420edff49b2a3137bfd51785d05.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,312 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/847047970a7342ca9f8e0ddd18892727.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,313 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/f0615b5d076841bca4178af84f38ad9a.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,314 DEBUG [StoreCloser-testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/ae4593f8ead64db79efd0ed0d244e59e.941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,316 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-11 03:55:56,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:56,317 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451. 2023-02-11 03:55:56,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e29181dea4ebd71fa849a2f526b3b451: Waiting for close lock at 1676087756298Running coprocessor pre-close hooks at 1676087756298Disabling compacts and flushes for region at 1676087756298Disabling writes for close at 1676087756298Writing region close event to WAL at 1676087756314 (+16 ms)Running coprocessor post-close hooks at 1676087756317 (+3 ms)Closed at 1676087756317 2023-02-11 03:55:56,318 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:56,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=e29181dea4ebd71fa849a2f526b3b451, regionState=CLOSED 2023-02-11 03:55:56,318 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087756318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087756318"}]},"ts":"1676087756318"} 2023-02-11 03:55:56,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-11 03:55:56,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure e29181dea4ebd71fa849a2f526b3b451, server=jenkins-hbase4.apache.org,33925,1676087644887 in 192 msec 2023-02-11 03:55:56,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-11 03:55:56,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e29181dea4ebd71fa849a2f526b3b451, UNASSIGN in 197 msec 2023-02-11 03:55:56,321 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087756321"}]},"ts":"1676087756321"} 2023-02-11 03:55:56,322 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-11 03:55:56,323 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-11 03:55:56,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 207 msec 2023-02-11 03:55:56,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-11 03:55:56,422 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-11 03:55:56,422 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-11 03:55:56,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,424 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,425 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 03:55:56,428 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,428 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:56,428 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:56,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/recovered.edits] 2023-02-11 03:55:56,430 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/recovered.edits] 2023-02-11 03:55:56,430 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2, FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/recovered.edits] 2023-02-11 03:55:56,433 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/0eb6ce937dd449fbb86df9a70c8b1ac1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf1/0eb6ce937dd449fbb86df9a70c8b1ac1 2023-02-11 03:55:56,433 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/9ec9706bcc34406e8ec478e4ffa304aa to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf1/9ec9706bcc34406e8ec478e4ffa304aa 2023-02-11 03:55:56,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1390d400dda84c96b763e4650c273b98 2023-02-11 03:55:56,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/1afdb9bc3986411ba39c4cff6def5c0f 2023-02-11 03:55:56,435 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/bfdcc4331fb54a85ba0abf7a5d060d91 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/cf2/bfdcc4331fb54a85ba0abf7a5d060d91 2023-02-11 03:55:56,435 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/14e946de06884295a346074b907fd411 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/cf2/14e946de06884295a346074b907fd411 2023-02-11 03:55:56,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/28ce3f4f82c34e4c9315c0a68865c6b2 2023-02-11 03:55:56,437 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/3acd20ba935a4459b53a1674ffa3f0a1 2023-02-11 03:55:56,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451/recovered.edits/90.seqid 2023-02-11 03:55:56,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/recovered.edits/90.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c/recovered.edits/90.seqid 2023-02-11 03:55:56,439 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/e29181dea4ebd71fa849a2f526b3b451 2023-02-11 03:55:56,439 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/74e1d76b732588b92d3ddfdecf8ade4c 2023-02-11 03:55:56,439 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/7f8149e35be54518a65375dd2b693659 2023-02-11 03:55:56,440 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/dc8eac6ae08b4e3e84f75a3b1e845b28 2023-02-11 03:55:56,441 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf1/ed43a5800e0442478419adcc729cb4ec 2023-02-11 03:55:56,443 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/1ba5936892f3464281b78f79f8465110 2023-02-11 03:55:56,445 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/6b60de29d6ae4c8ebdd89b6ae291f2f6 2023-02-11 03:55:56,446 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/847047970a7342ca9f8e0ddd18892727 2023-02-11 03:55:56,447 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ae4593f8ead64db79efd0ed0d244e59e 2023-02-11 03:55:56,448 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05 to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/e8878420edff49b2a3137bfd51785d05 2023-02-11 03:55:56,449 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/ee9818bb91284cb1b503a67eaac2764e 2023-02-11 03:55:56,450 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/cf2/f0615b5d076841bca4178af84f38ad9a 2023-02-11 03:55:56,452 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/recovered.edits/85.seqid to hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/archive/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22/recovered.edits/85.seqid 2023-02-11 03:55:56,452 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/.tmp/data/default/testSplitTableRegion/941c0ecb516230bdc53f83554817cc22 2023-02-11 03:55:56,452 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-11 03:55:56,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,456 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-11 03:55:56,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-11 03:55:56,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-11 03:55:56,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087756458"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:56,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087756458"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:56,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087756458"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:56,459 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:55:56,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 941c0ecb516230bdc53f83554817cc22, NAME => 'testSplitTableRegion,,1676087752686.941c0ecb516230bdc53f83554817cc22.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e29181dea4ebd71fa849a2f526b3b451, NAME => 'testSplitTableRegion,,1676087754232.e29181dea4ebd71fa849a2f526b3b451.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 74e1d76b732588b92d3ddfdecf8ade4c, NAME => 'testSplitTableRegion,41,1676087754232.74e1d76b732588b92d3ddfdecf8ade4c.', STARTKEY => '41', ENDKEY => ''}] 2023-02-11 03:55:56,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-11 03:55:56,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087756460"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:56,461 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-11 03:55:56,462 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-11 03:55:56,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 40 msec 2023-02-11 03:55:56,526 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36397] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-11 03:55:56,526 INFO [Listener at localhost/45015] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-11 03:55:56,540 INFO [Listener at localhost/45015] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374 (was 371) - Thread LEAK? -, OpenFileDescriptor=825 (was 838), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=109 (was 118), ProcessCount=168 (was 168), AvailableMemoryMB=2896 (was 2949) 2023-02-11 03:55:56,540 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 03:55:56,540 INFO [Listener at localhost/45015] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:56,541 DEBUG [Listener at localhost/45015] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x328d140f to 127.0.0.1:55824 2023-02-11 03:55:56,542 DEBUG [Listener at localhost/45015] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,542 DEBUG [Listener at localhost/45015] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 03:55:56,542 DEBUG [Listener at localhost/45015] util.JVMClusterUtil(257): Found active master hash=316987840, stopped=false 2023-02-11 03:55:56,543 INFO [Listener at localhost/45015] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:55:56,545 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:55:56,545 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:55:56,545 INFO [Listener at localhost/45015] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:55:56,545 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:55:56,545 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:55:56,545 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:55:56,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:55:56,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:55:56,546 DEBUG [Listener at localhost/45015] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x10c3821d to 127.0.0.1:55824 2023-02-11 03:55:56,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:55:56,546 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:55:56,546 DEBUG [Listener at localhost/45015] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,546 INFO [Listener at localhost/45015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36661,1676087644839' ***** 2023-02-11 03:55:56,546 INFO [Listener at localhost/45015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 03:55:56,546 INFO [Listener at localhost/45015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33925,1676087644887' ***** 2023-02-11 03:55:56,546 INFO [Listener at localhost/45015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 03:55:56,547 INFO [Listener at localhost/45015] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36767,1676087644919' ***** 2023-02-11 03:55:56,547 INFO [Listener at localhost/45015] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 03:55:56,547 INFO [RS:1;jenkins-hbase4:33925] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 03:55:56,547 INFO [RS:0;jenkins-hbase4:36661] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 03:55:56,547 INFO [RS:2;jenkins-hbase4:36767] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 03:55:56,547 INFO [RS:0;jenkins-hbase4:36661] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 03:55:56,547 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 03:55:56,547 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 03:55:56,547 INFO [RS:0;jenkins-hbase4:36661] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 03:55:56,547 INFO [RS:2;jenkins-hbase4:36767] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 03:55:56,547 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:56,547 INFO [RS:2;jenkins-hbase4:36767] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 03:55:56,547 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 03:55:56,547 DEBUG [RS:0;jenkins-hbase4:36661] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d2905d0 to 127.0.0.1:55824 2023-02-11 03:55:56,548 INFO [RS:1;jenkins-hbase4:33925] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 03:55:56,548 DEBUG [RS:0;jenkins-hbase4:36661] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,548 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:56,548 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36661,1676087644839; all regions closed. 2023-02-11 03:55:56,548 DEBUG [RS:2;jenkins-hbase4:36767] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77c8c135 to 127.0.0.1:55824 2023-02-11 03:55:56,548 DEBUG [RS:2;jenkins-hbase4:36767] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,548 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36767,1676087644919; all regions closed. 2023-02-11 03:55:56,548 INFO [RS:1;jenkins-hbase4:33925] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 03:55:56,548 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(3304): Received CLOSE for af122cb86300c9588537dca293c1ed53 2023-02-11 03:55:56,549 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:56,549 DEBUG [RS:1;jenkins-hbase4:33925] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1f9b5be4 to 127.0.0.1:55824 2023-02-11 03:55:56,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing af122cb86300c9588537dca293c1ed53, disabling compactions & flushes 2023-02-11 03:55:56,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:55:56,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:55:56,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. after waiting 0 ms 2023-02-11 03:55:56,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:55:56,549 DEBUG [RS:1;jenkins-hbase4:33925] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing af122cb86300c9588537dca293c1ed53 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-11 03:55:56,549 INFO [RS:1;jenkins-hbase4:33925] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 03:55:56,549 INFO [RS:1;jenkins-hbase4:33925] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 03:55:56,549 INFO [RS:1;jenkins-hbase4:33925] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 03:55:56,549 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 03:55:56,550 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-11 03:55:56,550 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1479): Online Regions={af122cb86300c9588537dca293c1ed53=hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53., 1588230740=hbase:meta,,1.1588230740} 2023-02-11 03:55:56,550 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1505): Waiting on 1588230740, af122cb86300c9588537dca293c1ed53 2023-02-11 03:55:56,551 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 03:55:56,551 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 03:55:56,551 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 03:55:56,551 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 03:55:56,551 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 03:55:56,552 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-11 03:55:56,561 DEBUG [RS:0;jenkins-hbase4:36661] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs 2023-02-11 03:55:56,561 INFO [RS:0;jenkins-hbase4:36661] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36661%2C1676087644839:(num 1676087646599) 2023-02-11 03:55:56,561 DEBUG [RS:0;jenkins-hbase4:36661] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,561 INFO [RS:0;jenkins-hbase4:36661] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:56,561 DEBUG [RS:2;jenkins-hbase4:36767] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs 2023-02-11 03:55:56,561 INFO [RS:0;jenkins-hbase4:36661] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 03:55:56,561 INFO [RS:2;jenkins-hbase4:36767] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36767%2C1676087644919:(num 1676087646599) 2023-02-11 03:55:56,562 DEBUG [RS:2;jenkins-hbase4:36767] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:56,562 INFO [RS:2;jenkins-hbase4:36767] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:56,562 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:55:56,562 INFO [RS:2;jenkins-hbase4:36767] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 03:55:56,562 INFO [RS:0;jenkins-hbase4:36661] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 03:55:56,562 INFO [RS:0;jenkins-hbase4:36661] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 03:55:56,562 INFO [RS:0;jenkins-hbase4:36661] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 03:55:56,568 INFO [RS:2;jenkins-hbase4:36767] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 03:55:56,568 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:55:56,568 INFO [RS:2;jenkins-hbase4:36767] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 03:55:56,568 INFO [RS:0;jenkins-hbase4:36661] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36661 2023-02-11 03:55:56,568 INFO [RS:2;jenkins-hbase4:36767] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 03:55:56,569 INFO [RS:2;jenkins-hbase4:36767] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36767 2023-02-11 03:55:56,722 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:56,722 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:56,722 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36661,1676087644839 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:56,728 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@44ecde17 rejected from java.util.concurrent.ThreadPoolExecutor@40d35252[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,728 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:55:56,728 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@16a01f9d rejected from java.util.concurrent.ThreadPoolExecutor@40d35252[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,729 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:56,729 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6e05e23b rejected from java.util.concurrent.ThreadPoolExecutor@40d35252[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,729 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@79bcb395 rejected from java.util.concurrent.ThreadPoolExecutor@7d83287a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1374) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,729 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:55:56,729 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@154fdea7 rejected from java.util.concurrent.ThreadPoolExecutor@7d83287a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,729 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36767,1676087644919 2023-02-11 03:55:56,729 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6d57de2c rejected from java.util.concurrent.ThreadPoolExecutor@7d83287a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,730 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36767,1676087644919] 2023-02-11 03:55:56,730 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36767,1676087644919; numProcessing=1 2023-02-11 03:55:56,735 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36767,1676087644919 already deleted, retry=false 2023-02-11 03:55:56,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/.tmp/info/704a393903f044bdaab73966691e7b77 2023-02-11 03:55:56,735 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36767,1676087644919 expired; onlineServers=2 2023-02-11 03:55:56,735 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36661,1676087644839] 2023-02-11 03:55:56,735 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36661,1676087644839; numProcessing=2 2023-02-11 03:55:56,736 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36661,1676087644839 already deleted, retry=false 2023-02-11 03:55:56,736 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36661,1676087644839 expired; onlineServers=1 2023-02-11 03:55:56,738 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/info/7909474b50174df384ab9e4a1102b56e 2023-02-11 03:55:56,740 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/.tmp/info/704a393903f044bdaab73966691e7b77 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/info/704a393903f044bdaab73966691e7b77 2023-02-11 03:55:56,742 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7909474b50174df384ab9e4a1102b56e 2023-02-11 03:55:56,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/info/704a393903f044bdaab73966691e7b77, entries=2, sequenceid=6, filesize=4.8 K 2023-02-11 03:55:56,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for af122cb86300c9588537dca293c1ed53 in 197ms, sequenceid=6, compaction requested=false 2023-02-11 03:55:56,751 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1505): Waiting on 1588230740, af122cb86300c9588537dca293c1ed53 2023-02-11 03:55:56,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/namespace/af122cb86300c9588537dca293c1ed53/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-11 03:55:56,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:56,755 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:55:56,755 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for af122cb86300c9588537dca293c1ed53: Waiting for close lock at 1676087756549Running coprocessor pre-close hooks at 1676087756549Disabling compacts and flushes for region at 1676087756549Disabling writes for close at 1676087756549Obtaining lock to block concurrent updates at 1676087756549Preparing flush snapshotting stores in af122cb86300c9588537dca293c1ed53 at 1676087756549Finished memstore snapshotting hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676087756549Flushing stores of hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. at 1676087756550 (+1 ms)Flushing af122cb86300c9588537dca293c1ed53/info: creating writer at 1676087756550Flushing af122cb86300c9588537dca293c1ed53/info: appending metadata at 1676087756558 (+8 ms)Flushing af122cb86300c9588537dca293c1ed53/info: closing flushed file at 1676087756558Flushing af122cb86300c9588537dca293c1ed53/info: reopening flushed file at 1676087756741 (+183 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for af122cb86300c9588537dca293c1ed53 in 197ms, sequenceid=6, compaction requested=false at 1676087756746 (+5 ms)Writing region close event to WAL at 1676087756748 (+2 ms)Running coprocessor post-close hooks at 1676087756753 (+5 ms)Closed at 1676087756755 (+2 ms) 2023-02-11 03:55:56,756 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676087647020.af122cb86300c9588537dca293c1ed53. 2023-02-11 03:55:56,759 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/rep_barrier/33e636168739474bb5ac8059e2a51f4a 2023-02-11 03:55:56,762 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33e636168739474bb5ac8059e2a51f4a 2023-02-11 03:55:56,830 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:56,830 INFO [RS:2;jenkins-hbase4:36767] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36767,1676087644919; zookeeper connection closed. 2023-02-11 03:55:56,830 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@73349cb6 rejected from java.util.concurrent.ThreadPoolExecutor@40d35252[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,830 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36767-0x1004183b4960003, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:56,830 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@75012ad4 rejected from java.util.concurrent.ThreadPoolExecutor@40d35252[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,830 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@6f549381] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@6f549381 2023-02-11 03:55:56,930 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:56,930 INFO [RS:0;jenkins-hbase4:36661] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36661,1676087644839; zookeeper connection closed. 2023-02-11 03:55:56,930 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@337f11b5 rejected from java.util.concurrent.ThreadPoolExecutor@7d83287a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,930 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:36661-0x1004183b4960001, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:56,930 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7653b39b rejected from java.util.concurrent.ThreadPoolExecutor@7d83287a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 309] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:56,930 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1dc9a9e9] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1dc9a9e9 2023-02-11 03:55:56,951 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 03:55:57,072 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:57,151 DEBUG [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 03:55:57,172 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/table/7831945491364efea0598ba42b996d3c 2023-02-11 03:55:57,174 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7831945491364efea0598ba42b996d3c 2023-02-11 03:55:57,175 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/info/7909474b50174df384ab9e4a1102b56e as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/info/7909474b50174df384ab9e4a1102b56e 2023-02-11 03:55:57,178 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7909474b50174df384ab9e4a1102b56e 2023-02-11 03:55:57,178 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/info/7909474b50174df384ab9e4a1102b56e, entries=59, sequenceid=210, filesize=11.1 K 2023-02-11 03:55:57,178 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/rep_barrier/33e636168739474bb5ac8059e2a51f4a as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/rep_barrier/33e636168739474bb5ac8059e2a51f4a 2023-02-11 03:55:57,181 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33e636168739474bb5ac8059e2a51f4a 2023-02-11 03:55:57,181 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/rep_barrier/33e636168739474bb5ac8059e2a51f4a, entries=24, sequenceid=210, filesize=7.5 K 2023-02-11 03:55:57,181 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/.tmp/table/7831945491364efea0598ba42b996d3c as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/table/7831945491364efea0598ba42b996d3c 2023-02-11 03:55:57,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7831945491364efea0598ba42b996d3c 2023-02-11 03:55:57,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/table/7831945491364efea0598ba42b996d3c, entries=36, sequenceid=210, filesize=8.0 K 2023-02-11 03:55:57,185 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 634ms, sequenceid=210, compaction requested=false 2023-02-11 03:55:57,188 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-11 03:55:57,189 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-11 03:55:57,189 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 03:55:57,190 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 03:55:57,190 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676087756551Running coprocessor pre-close hooks at 1676087756551Disabling compacts and flushes for region at 1676087756551Disabling writes for close at 1676087756551Obtaining lock to block concurrent updates at 1676087756553 (+2 ms)Preparing flush snapshotting stores in 1588230740 at 1676087756553Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676087756553Flushing stores of hbase:meta,,1.1588230740 at 1676087756555 (+2 ms)Flushing 1588230740/info: creating writer at 1676087756556 (+1 ms)Flushing 1588230740/info: appending metadata at 1676087756563 (+7 ms)Flushing 1588230740/info: closing flushed file at 1676087756568 (+5 ms)Flushing 1588230740/rep_barrier: creating writer at 1676087756742 (+174 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676087756745 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676087756746 (+1 ms)Flushing 1588230740/table: creating writer at 1676087756762 (+16 ms)Flushing 1588230740/table: appending metadata at 1676087756765 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676087756765Flushing 1588230740/info: reopening flushed file at 1676087757175 (+410 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676087757179 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676087757182 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 634ms, sequenceid=210, compaction requested=false at 1676087757185 (+3 ms)Writing region close event to WAL at 1676087757186 (+1 ms)Running coprocessor post-close hooks at 1676087757189 (+3 ms)Closed at 1676087757190 (+1 ms) 2023-02-11 03:55:57,190 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 03:55:57,351 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33925,1676087644887; all regions closed. 2023-02-11 03:55:57,356 DEBUG [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs 2023-02-11 03:55:57,357 INFO [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33925%2C1676087644887.meta:.meta(num 1676087646747) 2023-02-11 03:55:57,361 DEBUG [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/oldWALs 2023-02-11 03:55:57,361 INFO [RS:1;jenkins-hbase4:33925] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33925%2C1676087644887:(num 1676087646599) 2023-02-11 03:55:57,361 DEBUG [RS:1;jenkins-hbase4:33925] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:57,361 INFO [RS:1;jenkins-hbase4:33925] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:55:57,361 INFO [RS:1;jenkins-hbase4:33925] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-11 03:55:57,361 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:55:57,362 INFO [RS:1;jenkins-hbase4:33925] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33925 2023-02-11 03:55:57,363 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33925,1676087644887 2023-02-11 03:55:57,363 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:55:57,363 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@20b703e5 rejected from java.util.concurrent.ThreadPoolExecutor@2488832b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:57,365 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33925,1676087644887] 2023-02-11 03:55:57,365 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33925,1676087644887; numProcessing=3 2023-02-11 03:55:57,366 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33925,1676087644887 already deleted, retry=false 2023-02-11 03:55:57,366 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33925,1676087644887 expired; onlineServers=0 2023-02-11 03:55:57,366 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,36397,1676087643676' ***** 2023-02-11 03:55:57,367 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 03:55:57,367 DEBUG [M:0;jenkins-hbase4:36397] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@53d9d3fb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:55:57,367 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,36397,1676087643676 2023-02-11 03:55:57,367 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,36397,1676087643676; all regions closed. 2023-02-11 03:55:57,367 DEBUG [M:0;jenkins-hbase4:36397] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:57,367 DEBUG [M:0;jenkins-hbase4:36397] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 03:55:57,367 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 03:55:57,367 DEBUG [M:0;jenkins-hbase4:36397] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 03:55:57,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087646170] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087646170,5,FailOnTimeoutGroup] 2023-02-11 03:55:57,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087646170] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087646170,5,FailOnTimeoutGroup] 2023-02-11 03:55:57,368 INFO [M:0;jenkins-hbase4:36397] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 03:55:57,368 INFO [M:0;jenkins-hbase4:36397] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 03:55:57,368 INFO [M:0;jenkins-hbase4:36397] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 03:55:57,368 DEBUG [M:0;jenkins-hbase4:36397] master.HMaster(1502): Stopping service threads 2023-02-11 03:55:57,368 INFO [M:0;jenkins-hbase4:36397] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 03:55:57,368 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 03:55:57,368 ERROR [M:0;jenkins-hbase4:36397] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] 2023-02-11 03:55:57,368 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-11 03:55:57,368 INFO [M:0;jenkins-hbase4:36397] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:55:57,368 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 03:55:57,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:55:57,369 DEBUG [M:0;jenkins-hbase4:36397] zookeeper.ZKUtil(398): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 03:55:57,369 WARN [M:0;jenkins-hbase4:36397] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 03:55:57,369 INFO [M:0;jenkins-hbase4:36397] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:55:57,369 INFO [M:0;jenkins-hbase4:36397] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 03:55:57,369 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 03:55:57,369 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:55:57,369 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:55:57,369 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 03:55:57,369 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:55:57,369 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.91 KB heapSize=718.93 KB 2023-02-11 03:55:57,382 INFO [M:0;jenkins-hbase4:36397] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.91 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f56ce128a7e94f49b1f0b18a8a576aa6 2023-02-11 03:55:57,385 INFO [M:0;jenkins-hbase4:36397] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f56ce128a7e94f49b1f0b18a8a576aa6 2023-02-11 03:55:57,386 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f56ce128a7e94f49b1f0b18a8a576aa6 as hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f56ce128a7e94f49b1f0b18a8a576aa6 2023-02-11 03:55:57,388 INFO [M:0;jenkins-hbase4:36397] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f56ce128a7e94f49b1f0b18a8a576aa6 2023-02-11 03:55:57,388 INFO [M:0;jenkins-hbase4:36397] regionserver.HStore(1080): Added hdfs://localhost:39467/user/jenkins/test-data/ad700a5f-4dfc-4814-98d6-287a1cd7896b/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f56ce128a7e94f49b1f0b18a8a576aa6, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-11 03:55:57,389 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegion(2947): Finished flush of dataSize ~607.91 KB/622499, heapSize ~718.91 KB/736168, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false 2023-02-11 03:55:57,390 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:55:57,390 DEBUG [M:0;jenkins-hbase4:36397] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676087757369Disabling compacts and flushes for region at 1676087757369Disabling writes for close at 1676087757369Obtaining lock to block concurrent updates at 1676087757369Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676087757369Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622499, getHeapSize=736168, getOffHeapSize=0, getCellsCount=1499 at 1676087757370 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676087757370Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676087757370Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676087757373 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676087757373Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676087757386 (+13 ms)Finished flush of dataSize ~607.91 KB/622499, heapSize ~718.91 KB/736168, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 20ms, sequenceid=1282, compaction requested=false at 1676087757389 (+3 ms)Writing region close event to WAL at 1676087757390 (+1 ms)Closed at 1676087757390 2023-02-11 03:55:57,393 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:55:57,393 INFO [M:0;jenkins-hbase4:36397] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 03:55:57,394 INFO [M:0;jenkins-hbase4:36397] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36397 2023-02-11 03:55:57,400 DEBUG [M:0;jenkins-hbase4:36397] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,36397,1676087643676 already deleted, retry=false 2023-02-11 03:55:57,465 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:57,465 INFO [RS:1;jenkins-hbase4:33925] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33925,1676087644887; zookeeper connection closed. 2023-02-11 03:55:57,466 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@48910784 rejected from java.util.concurrent.ThreadPoolExecutor@2488832b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:57,466 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): regionserver:33925-0x1004183b4960002, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:57,466 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3576a8a8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3576a8a8 2023-02-11 03:55:57,466 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3050ddbc rejected from java.util.concurrent.ThreadPoolExecutor@2488832b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:57,466 INFO [Listener at localhost/45015] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-11 03:55:57,566 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:57,566 INFO [M:0;jenkins-hbase4:36397] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,36397,1676087643676; zookeeper connection closed. 2023-02-11 03:55:57,566 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@43209d53 rejected from java.util.concurrent.ThreadPoolExecutor@245b016c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:57,566 DEBUG [Listener at localhost/45015-EventThread] zookeeper.ZKWatcher(600): master:36397-0x1004183b4960000, quorum=127.0.0.1:55824, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:55:57,566 ERROR [Listener at localhost/45015-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@47ed878 rejected from java.util.concurrent.ThreadPoolExecutor@245b016c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:55:57,568 WARN [Listener at localhost/45015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 03:55:57,570 INFO [Listener at localhost/45015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:55:57,672 WARN [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 03:55:57,672 WARN [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 5cdfedec-baf9-457a-bd3c-897c5ed5a081) service to localhost/127.0.0.1:39467 2023-02-11 03:55:57,675 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data5/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,675 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data6/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,676 WARN [Listener at localhost/45015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 03:55:57,679 INFO [Listener at localhost/45015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:55:57,785 WARN [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 03:55:57,785 WARN [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid e31d1c49-fd88-49e6-ab6b-686fae2c4c65) service to localhost/127.0.0.1:39467 2023-02-11 03:55:57,787 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data3/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,788 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data4/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,789 WARN [Listener at localhost/45015] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 03:55:57,804 INFO [Listener at localhost/45015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:55:57,810 WARN [BP-511681113-172.31.14.131-1676087640548 heartbeating to localhost/127.0.0.1:39467] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-511681113-172.31.14.131-1676087640548 (Datanode Uuid 328ac628-71ce-4e94-9ad2-68da08fa646b) service to localhost/127.0.0.1:39467 2023-02-11 03:55:57,813 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data1/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,813 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/647393ef-f9c1-436d-684d-96e18d7feef0/cluster_0414f093-7cd0-f4ae-b6c6-ab8f93e64544/dfs/data/data2/current/BP-511681113-172.31.14.131-1676087640548] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:55:57,839 INFO [Listener at localhost/45015] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:55:58,028 INFO [Listener at localhost/45015] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 03:55:58,253 INFO [Listener at localhost/45015] hbase.HBaseTestingUtility(1293): Minicluster is down